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ABSTRACT 

The  purpose  of  this  thesis  is  to  study  and  enhance  an  automatic  image  feature 
analysis  system  for  aerial  photographs  in  order  to  detect  changes  with  respect  to  a 
model  defined  in  a  knowledge-base.  The  system  interpretation  must  resemble  some 
intelligent  human  interpretation  and  by  its  use  should  be  able  to  reduce  the  manual 
effort  in   processing   large  volumes  of  data. 

This  computerized  rule-based  system  is  integrated  by  using  Prolog  and  Image 
processing  operators,  which  run  in  a  multiprocess  environment  and  communicate 
through  a  blackboard  storage.  The  Prolog  expert  provides  the  model  through  a 
collection  of  certainty  rules  and  facts,  and  uses  its  "inference  engine"  capability  to 
combine  different  measurements.  They  are  obtained  as  the  evidence  necessary  to 
deduce  a  conclusion  about  the  condition  of  a  feature. 

The  system  must  be  able  to  deal  with  uncertainties  generated  by  noise  in  the 
images,  variability  of  imaging  conditions,  and  possible  errors  in  the  model.  The  main 
task  is  the  formulation  of  interpretation  rules  so  that  the  expert  system  can  mimic  the 
"reasoning"  effectively  from  the  domain  principles  and  yield  high  confidence  results. 

A  system  that  satisfies  some  of  the  stated  specifications  has  already  been 
developed  by  Prof.  Chin-Hwa  Lee  and  Hsi-Jian  Lee.  This  thesis  as  the  continuation  of 
the  investigation  process  addresses  the  following  objectives: 

1.  Analvze  in-depth  all  the  Fortran  programs  in  order  to  detect  possible  errors  in 
the  Lmage  Processing  operators  (The  main  modules  are:  Shadow  Processing. 
Model  guided  Split" and  Merge.  Region  Selection  and  Model  Projection!: 
additionally  the  documentation  of  all  the  programs  should  be  generated. 

2.  Expand  the  Rule  Data  base  and  Facts  Data  base  to  enhance  the  actual 
capabilities  of  the  expert  interpretation  system. 

3.  Measure  the  CPU  time  for  each  of  the  subprocesses  in  order  to  determine  the 
time  consuming  parts  of  the  system.  New  approaches  to  improve  those  parts 
should  be  consTdered  and  developed. 

4.  Apply  the  svstem  to  different  kinds  of  images  to  evaluate  the  system  response 
to  several  illumination  conditions  and  diiferent  image  detail. 

5.  Develop  and  Incorporate  reflectance  modelling  capabilities  to  make  the  svstem 
more  accurate. 


THESIS  DISCLAIMER 

The  reader  is  cautioned  that  computer  programs  developed  in  this  research  may 
not  have  been  exercised  for  all  cases  of  interest.  While  every  effort  has  been  made, 
within  the  time  available,  to  ensure  that  the  programs  are  free  of  computational  and 
logic  errors,  they  cannot  be  considered  validated.  Any  application  of  these  programs 
without  additional  verification  is  at  the  risk,  of  the  user. 
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I.  INTRODUCTION 

A.       THE  BACKGROUND 

To  have  automatic  image  feature  analysis  capabilities  over  an  aerial  photograph, 
involves  locating  and  studying  specific  objects.  In  this  analysis,  locating  buildings  and 
detecting  changes  in  their  structure  is  the  objective.  These  are  difficult  tasks  because  of 
the  variability  in  image  conditions  such  as  weather,  time  and  attitude  when  the 
photograph  was  taken.  Other  possible  reasons  are  the  introduction  of  noise  in  the 
digitized  image,  errors  in  the  'model'  used  to  match  the  observations,  and  errors  in  the 
results  generated  by  the  Change  Detection  process  itself. 

There  have  been  several  approaches  to  generate  model-based  image 
interpretation  systems,  see  (Refs.  1,2,3).  These  efforts  have  tended  to  concentrate  on 
high-level  issues  such  as  three  dimensional  object  representation  and  matching.  Others 
have  been  involved  in  aerial  photograph  analysis,  see  (Refs.  4.5.6).  The  need  to  have 
automatic  image  interpretation  systems  is  related  directly  to  the  large  number  of 
images  obtained  from  satellites  and  intelligence  sources.  Updating  the  information 
provided  by  these  photographs  requires  the  employment  of  human  resources  and  time 
to  analyze,  detect  and  identify  changes  in  existing  buildings  or  to  establish  the  creation 
of  new  ones.  Because  this  process  doesn't  guarantee  its  results  to  be  free  of  errors,  it  is 
time  consuming  and  takes  large  amounts  of  human  resources.  Consequently  the  use  of 
an  automatic  Image  Interpretation  System  is  justified. 

If  an  Expert  System  could  be  created  to  perform  automatically  the  image  feature 
interpretation,  the  same  process  could  also  be  used  to  identify  other  specified  features. 
Its  principles  could  be  applied  to  a  wider  number  of  applications  like  the  development 
of  artificial  vision  systems.  The  system  developed  by  Dr.  Chin-Hwa  Lee  and  Hsi-Jian 
Lee  actually  satisfy  the  main  objectives  of  this  investigation.  The  thesis  elfort  should 
be  oriented  to  continue  the  implementation  of  new  capabilities  and  to  make  this  system 
more  efficient  in  terms  of  execution  speed  and  memory  usage.  To  obtain  these 
objectives,  is  necessary  to  review  the  software  in  detail  and  obtain  a  better 
understanding  of  how  the  system  works  in  order  to  point  out;  the  possible  areas  of 
enhancement.  The  work  to  be  done  here  will  be  considered  as  the  necessary  steps  to  a 
better  and  more  sophisticated  method  of  image  interpretation. 
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The  contents  of  this  thesis  include: 

Chapter  I:  A  brief  description  of  the  problem  and  advantages  of  using  expert  systems 
for  the  implementation  of  the  Image  Interpretation. 

Chapter  II  :  A  description  of  how  an  expert  system  could  be  created  to  perform  the 
required  objectives,  also  has  a  general  overview  of  the  actual  system  structure. 

Chapter  III   :   Some  concepts  on  photogrammetry.  This  background  is  required  for 
studying  the  system,  because  the  analysis  is  done  on  aerial  photographs. 

Chapter  IV  :  Shows  the  work  done  through  the  development  of  this  thesis  and  includes 
details  on  the  execution  of  predetermined  tasks. 

Chapter    V    :      Gives    the    conclusions    obtained    after    working    with    the    Image 
Interpretation  System. 

B.       EXPERT  SYSTEM  IMPLEMENTATION 

Expert  systems  are  computer  programs  that  emulate  the  behavior  of  human 
experts  in  a  specified  domain  of  knowledge,  they  use  knowledge  and  inference 
procedures  to  solve  problems  that  are  difficult  enough  and  require  significant  human 
expertise  for  their  solutions,  see  (Ref.  7). 

Some  capabilities  found  in  the  expert  systems  are  needed  to  develop  our  image 
interpretation  system  as  the  followings:  they  are  'evovable',  that  is,  easy  to  modify  and 
update,  they  also  follow  a  top-down  modular  design  approach  where  modules  are  used 
to  hide  details  and  through  their  use  later  changes  in  the  design  are  easy  to  make  thus 
keeping  the  knowledge  base  complete  and  accurate.  They  have  the  capability  of  both 
forward  and  backward  reasoning.  Forward  chaining  is  a  problem  solving  paradigm  in 
which  chaining  starts  from  a  set  of  conditions  and  moves  toward  some  conclusion. 
Backward  chaining  is  a  process  that  requires  a  goal  or  a  hypothesis  as  a  starting  point. 
from  where  the  process  proceed  'working  backwards'  along  a  path  to  determine  if  the 
hypothesis  is  true.  Improvement  in  the  performance  of  the  program  is  possible  by 
constraining  the  search  for  a  solution.  Additionally  expert  systems  allow  uncertainties 
to  be  taken  into  account  during  the  design  of  the  system. 

The  structure  of  the  Image  Interpretation  system  consists  of  knowledge  clauses: 
rules,  facts  and  structures;  a  reasoning  element:  deductive  inference  engine  and  control 
programs;  and  image  processing  routines:  to  process  the  image  and  work  as  the 
computational  environment. 


The  knowledge  base  approach  allows  us  to  use  the  expert  knowledge  of  the 
domain  by  developing  change  detection  rules  and  incorporating  a-priori  knowledge,  see 
(Ref.  S).  The  Rule  Base  is  an  open  ended  structure  which  permits  information  to  be 
added  or  subtracted  in  an  incremental  manner.  These  are  tasks  performed  as  a 
consequence  of  finding  mistakes  in  the  process  and  improving  the  organization  of  the 
rules,  see  (Ref.  9).  These  changes  must  be  done  in  the  contents  of  the  control  structure 
in  an  efficient  manner,  therefore  the  system  must  have  a  modular  implementation  to 
facilitate  evolutionary  changes.  The  main  task  is  to  provide  the  system  with  the 
knowledge  of  expertise  in  the  field  of  image  interpretation,  then  the  computer  makes 
use  of  this  knowledge,  as  in  this  system,  to  detect  changes  in  buildings  located  in  aerial 
photographs. 

Expert  systems  software  show  greater  separation  between  data  and  program 
constructs.  This  feature  is  generated  by  providing  non-procedural  constructs  at  higher 
levels.  This  separation  enables  more  flexibility  in  the  creation  and  maintenance  of 
software  as  well  as  ease  of  understanding,  see  (Ref.  10).  An  expert  system  also  has  the 
following  three  characteristics  that  makes  it  convenient  for  our  application.  First,  an 
expert  has  knowledge  of  the  concepts  relevant  to  the  domain  and  can  solve  problems 
by  reasoning  from  the  domain  principles.  It  is  able  to  capture  and  retain  expertise 
from  an  expert  in  the  field,  which  will  be  used  in  the  interpretation  of  the  results. 
Second,  because  knowledge-based  systems  attempt  to  represent  expertise  explicitly,  the 
system  makes  decisions,  or  suggest  decisions,  and  explain  its  conclusions.  This  explicit 
representation  of  symbolic  structures,  behavior  and  reasoning  allows  flexibility  and 
adaptability  as  a  result  of  the  explicit  declarative  nature  of  the  representation.  Third, 
the  knowledge  representation  and  use  are  embeded  in  the  program's  control  How.  In 
contrast  to  traditional  procedural  programming  languages,  such  as  Fortran  and  Pascal, 
that  have  been  used  previously  to  implement  similar  systems,  these  systems,  even  being 
efficient,  lack  transparency,  modularity  and  flexibility.  Consequently  adding,  deleting 
or  updating  the  knowledge  is  a  difficult  task. 
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II.  SYSTEM  GENERATION 

A.   THE  APPROACH  TO  SOLVE  THE  PROBLEM 

Analysis  of  aerial  photographs  is  a  complex  task.  A  prime  cause  is  the  presence 
of  fine  texture  which  causes  difficulties  for  the  low  level  processes  such  as  the 
segmentation  routines,  see  (Refs.  11,4).  Another  source  of  difficulty  is  that  the  desired 
objects  and  structures  may  be  too  small  compared  to  the  resolution  of  the  photograph 
see  (Refs.  12,13).  A  detailed  analysis  of  a  complete  high  resolution  aerial  image  is 
generally  prohibitive  because  of  the  computational  costs. 

In  order  to  overcome  the  difficulties  in  the  brute  force  analysis  of  aerial 
photographs  and  obtain  a  better  description  of  the  area  under  consideration,  we  are 
using  artificial  intelligence  techniques.  The  system  must  emphasize  a  goal-directed 
approach  and  reasoning  techniques  with  possibilities  of  using  a-priori  knowledge  from 
the  'model'  and  data  calculated  from  the  image  to  achieve  the  determined  goals. 

The  approach  to  solve  the  change  detection  problem,  demands  the  establishment 
of  a  'model'  of  the  object,  which  will  be  used  as  the  guide  element,  a  Rule  Base  expert 
that  has  control  structure  for  the  Chanse  Detection  Svstem,  and  Imaae  Processins 
operators  that  provide  the  necessary  measurements  required  to  achieve  goals  of  the 
Prolog  Interpreter. 

The  term  'model'  will  be  used  to  describe  the  building  or  group  of  buildings  that 
we  are  trying  to  verify  in  the  image.  The  facts  database  used  as  the  environmental 
'model'  is  a  tree  like  structure  composed  of  a  number  of  nodes,  where  each  node  is 
described  uniquely,  see  (Table  1).  The  environmental  'model'  has  a  hierarchical 
structure  where  a-priori  information  describes  in  a  set  of  facts  about  their  individual 
properties  and  interrelationships.  Besides  the  facts  database,  there  must  be  several 
dynamic  databases  oriented  to  follow  the  development  of  the  process  and  to  record  the 
intermediate  changes  being  performed. 

Every  node  is  described  as  a  collection  of  related  properties.  It  includes 
appearance  knowledge  in  the  form  of  object  properties  :  predicted  location,  surface 
material  category,  children  and  ancestor  nodes,  minimum  boundary  rectangular, 
background  SMC,  and  properties  to  be  analized  in  this  node:  shadow,  size  and  tone, 
and    weights    assigned    to    these    properties.    The    height,    number    of   vertices,    and 
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TABLE  1 
MODEL  STRUCTURE 


HIERARCHY    LEVELS 


1. 


4 
5. 


Block 

Building     cluster 

Building 

Building     parts 

Subparts 


REPRESENTATION  OF  THE  'MODEL' 


BUILDINGS 


BLOCK  1 
BUILDING1 


BUILDING2 


UAVEQUARTER  ENLISTED-QUARTER         POST  OFFICE        COflKOEPT 


I 


UESTUING 


CENTRAL       EASTUING 


PARTI   PART2  PART3    PART  4   PART5   PARU   PART7   PARTS  PART?  PART  10 


16 


compactness  must  also  be  included  in  the  node  description.  The  facts  database  can 
specify  different  weights  for  the  assigned  properties  in  the  nodes.  Some  properties  are 
less  reliable,  conversely,  other  properties  are  considered  stable  and  therefore  given  more 
weight. 

The  control  structure,  in  this  case  the  Rule  Based  expert  exhibit  a  backward 
chaining  of  inference  as  it  tries  to  achieve  a  goal.  The  importance  of  this  process  relies 
on  the  rules  not  calling  to  each  other.  They  communicate  through  the  facts  in  the  path 
to  satisfy  a  goal,  and  make  use  of  newly  generated  assertions.  The  Rule  Based  expert 
has  relational  knowledge  in  the  form  of  parameterized  relations  between  objects.  It  can 
adapt  the  more  general  'model'  to  any  particular  image  by  a  reasoning  like  process. 
This  resemble   an  intelligent  human  interpretation  process. 

The  system  must  use  its  rules  to  combine  the  region  description  attributes  and 
select  thresholds  to  modify  its  own  results  according  to  the  image  conditions.  In  case  of 
failure,  the  set  of  rules  must  reason  about  the  situation,  tailor  its  behavior  to  redefine 
the  analysis  scheme  .and  obtain  better  results.  The  system  must  be  able  to  deal  with 
uncertainties  that  may  originate  from  the  imprecise  'model'  or  from  the  noise  data 
generated  by  the  image  processing  operators. 

The  Image  Processing  operator  also  provides  the  knowledge  base  with  measured 
relationships  obtained  from  the  segmented  image.  They  are:  the  labeled  region  map, 
number  of  regions  detected  in  the  image,  sun  direction,  base  plane  equation 
coefficients,  area,  perimeter,  average  tone  and  tone  variance,  minimum  boundary 
rectangle  and  compactness.  Also  included  are  general  information  of  the  image  as 
'affine'  coefficients,  exposure  station,  sun  direction  and  the  indicator  to  the 
corresponding  side  of  the  stereo  pair  of  images  being  analyzed. 

The  image  processing  operators  include  standard  image  processing  routines 
written  in  Fortran  or  Pascal  languages.  They  consist  of  region  segmentation 
algorithms,  shadow  processing,  and  region  analysis  capabilities.  The  recognition  of 
objects  are  done  in  several  processes  which  are  designed  to  detect  the  specific  type  of 
objects  in  a  prespecified  region.  In  this  system,  the  image  description  must  be 
measured  automatically  in  the  computational  environment  and  is  particular  to  the 
photograph  being  analyzed.  The  a-priori  knowledge  provides  the  Change  Detection 
System  with  predicted  area  facts,  that  helps  to  locate  the  area  of  the  buildings. 

The  Change  Detection  System  studies  the  correspondence  between  the  image"  and 
the  'model'  of  the  domain  of  interest,  which  in  this  case  describe  the  knowledse  about 
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buildings.  Once  the  'model'  has  been  consulted  by  the  Prolog  Interpreter,  the  process 
can  be  then  divided  in  two  main  parts:  the  first  is  the  segmentation  of  the  image  into 
regions,  where  their  properties  are  stored  into  the  background  interprocess 
communication  area  shared  by  all  subroutines  of  the  computational  environment;  the 
second  part  is  the  matching  and  inference  process. 

A  feature  being  analyzed  may  be  broken  into  numerous  region  segments,  see 
(Ref.  14)  not  necessarily  matching  the  predicted  model  projections.  Therefore,  the 
segmentation  will  be  crucial  to  the  success  of  the  system  behavior  and  additional 
efforts  will  be  required  to  refine  the  segmentation  of  the  image.  Here,  the  segmentation 
process  is  knowledge  directed  and  this  could  decide  the  success  or  failure  of  the 
analysis. 

To  summarize,  in  the  Change  Detection  System  there  are  three  levels  of 
interaction.  The  first  being  the  'model'  is  the  knowledge  base  that  provides  the  system 
with  the  building  description,  the  search  can  be  constrained  according  to  its  predicates. 
The  second  level,  being  the  Rule  Base  Process  is  the  one  that  decides  which  process  to 
initiate  to  satisfy  system  goals.  It  is  the  reasoning  part  that  tries  to  find  the  regions 
with  the  desired  characteristics.  The  Rule  Base  process  can  also  react  to  failure  by 
readapting  itself  and  trying  to  resatisfy  its  goals.  Finally,  the  third  level,  which  consists 
of  image  processing  operators,  examine  features  and  performs  computations  on  the 
image  itself. 

These  three  elements  interact  and  generate  the  evidence  necessary  to  deduce  a 
conclusion  reflecting  the  situation  of  a  feature.  After  evaluation,  if  the  calculated 
results  are  different  from  the  expected,  the  system  reevaluates  its  conditions  and  takes 
corrective  action. 

B.       HOW  THE  CHANGE  DETECTION  SYSTEM  WORKS 

After  the  facts  database  has  been  generated  for  the  available  buildings  and  the 
Rule  Base  expert  established  in  the  Change  Detection  System,  the  digitized  aerial 
photograph  is  segmented  into  regions  using  a  segmentation  algorithm.  The 
segmentation  algorithm  is  provided  by  the  Spider  library;  a  software  library  that 
contains  image  processing  operators.  The  characteristics  of  every  region  to  be  analized 
are  extracted.  Now.  by  using  the  building's  predicted  area  location  obtained  from  the 
facts  database,  its  position  is  verified  in  the  digitized  image  in  order  to  decide  wether  to 
continue    with    the    image    interpretation    analysis.      The    'blackboard'    interprocess 
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communication  area  is  created,  and  the  extracted  region  characteristics  are  also  read  by 

the  Change  Detection  System  to  be  included  in  the  facts  database. 

Now  the  Change  Detection  System  generates  a  list  of  nodes  as  subjects  of 
interest  to  establish  the  existence  of  the  buildings  and  to  obtain  the  partial  evidences 
necessary  to  deduce  a  general  conclusion.  The  nodes  in  the  list  (referenced  as 
'checkinglist')  will  be  activated  progressively  from  the  lowest  to  the  upper  hierarchy  in 
the  tree  structure. 

The  certainty  value  of  every  node  is  calculated  by  combining  the  weighted 
properties  of  the  node  and  multiplying  it  with  the  arithmetic  average  of  the  certainty 
value  from  all  the  children  nodes.  If  this  is  a  terminal  node,  its  father's  certainty 
should  be  updated  and  its  descendants  removed  from  the  'checkinglist'.  This  process 
continues  until  all  nodes  have  been  verified.  Once  the  certainty  value  of  a  node  is 
decided,  its  status  is  reported.  If  the  results  do  not  yield  enough  confidence  for  the 
existence  of  a  specified  node,  then  the  segmented  image  is  refined  by  a  'model'  guided 
split  and  merge  process.  The  split  and  merge  process  uses  the  corresponding  region 
associated  with  the  node  projection,  ft  implies  the  creation  of  new  regions  related  to 
the  segmented  areas  generated  by  overlapping  the  node  projection  over  the  labeled 
region  map.  After  the  refining  process,  the  'blackboard'  and  the  facts  database  are 
updated  with  the  new  region  characteristics. 

The  Prolog  expert  then  repeat  the  process  of  obtaining  the  certainty  values  of  the 
node  according  to  tone,  size,  shape  and  shadow  characteristics  in  the  following  ways: 

The  certainty  of  size  is  obtained  according  to  the  relation  of  node  projection  area 
against  the  sum  of  the  corresponding  region's  area. 

The  certainty  of  polygon  uses  the  mapped  area  of  the  node  against  the  sum  of  the 
intersecting  segments  in  the  labeled  region  map. 

The  certainty  of  shadow  is  obtained  by  first  obtaining  the  location  in  which  to 
look  for  the  shadow.  This  process  uses  implicit  knowledge  of  the  sun's  position  as 
provided  in  the  common  area  BOARD,  it  then  uses  the  image  processing  operators  to 
obtain  the  tones  and  areas  of  the  predicted  projection  shadow  areas,  and  evaluates 
them  against  the  ones  measured  from  the  digitized  image. 

The  certainty  of  tone  also  makes  use  of  the  sun's  position  and  image  conditions 
already  known  to  calculate  the  tone  of  the  regions  of  different  nodes.  It  uses  the 
surface  material  category  and  surface  normal  to  determine  the  amount  of  illumination 
reflected  from  the  node. 
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III.  PHOTOGRAMMETRY  TOPICS 

The  following  definitions  and  concepts  are  included  because  they  are  vital  to  the 
understanding  of  the  Change  Detection  System.  Many  of  the  parameters  discussed  here 
are  used  in  different  programs  and  subroutines  in  the  Change  Detection  System.  These 
concepts  have  been  summarized  from  a  text  in  photogrammetry.  see  (Ref.  15). 

A.       BASIC  GEOMETRY  OF  THE  PHOTOGRAPH.-  CONCEPTS 

1.  Orthographic  versus  Perspective  Projection 

A  map  is  a  reproduction  at  reduced  scale  of  an  orthographic  projection  from 
the  terrain  onto  a  reference  datum  plane.  A  perspective  or  central  projection,  is  one  in 
which  all  points  are  projected  onto  the  reference  plane  through  one  point  called  the 
perspective  center.  In  photography  the  sensitized  film  occupies  the  reference  plane 
where  the  negative  is  formed  behind  the  perspective  center,  see  (Fig.  3.1). 

2.  Image  and  Object  Spaces 

The  photograph  is  a  perspective  projection,  and  the  optical  center  of  the 
camera  lens  is  the  perspective  center.  The  region  between  the  perspective  center  and 
the  negative  of  the  photograph  is  called  the  image  space.  The  region  occupied  by  the 
object  to  be  photographed,  including  the  reference  coordinate  system,  is  called  the 
object  space. 

3.  Image  Coordinates 

The  plane  of  the  photograph,  which  contains  images  of  object  points,  is  a  2-D 
representation  of  a  generally  3-D  object  space.  The  X  and  Y  axes  of  the  photograph 
are  defined  by  the  fiducial  marks,  with  the  center  of  collimation  called  the  indicated 
principal  point  IPP  or  fiducial  center  taken  as  the  origin.  The  position  of  the 
perspective  center  will  be  over  the  IPP  at  a  focal  distance  T. 

4.  Object  Coordinates 

A  3-D  right  handed  Cartesian  Coordinate  System  X.  Y.  Z  is  used  as  reference 
for  the  object  space.  If  'L'  is  called  the  perspective  center,  then  the  object  coordinates 
of  the  perspective  center,  or  better  said  the  exposure  station  is  XL.  YL,  ZL  as 
described  in  the  PROJECT  and  XY2IJ  subroutines  see  Appendix  B  and  (Ref.  16). 
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The  region  between  the  perspective  center  and 
the  photograph  (actually  the  negative)  is 
called  the  ' inage  space'. 


Negative 


Perspective 


Z 


Y 


Y 


Object 
space 


_center 
Photograph. 


The  region  occupied  by  the  object  to  be  photographed 
including  the  reference  coordinate  systen  is 
called  the  'object  space'. 


X 


Figure  3.1     Image  and  object  spaces. 

5.   Interior  Orientation 

The  interior  orientation  of  a  photograph  defines  the  form  of  the  bundle  of 
rays  emerging  from  the  perspective  center  to  the  points  in  the  'image'  space.  It 
expresses  the  angular  relationship  between  object  space  rays  on  the  basis  of  the 
location  of  the  image  points  with  respect  to  the  perspective  center.  Thus  the  position 
of  the  perspective  center  with  respect  to  the  image  coordinate  system  represents  the 
geometric  elements  of  interior  orientation,  being  XO,  YO  and  F.  where  XO  and  YO  are 
the  2-D  coordinates  of  the  IPP. 
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6.  Exterior  Orientation 

The  exterior  orientation  of  the  photograph  defines  the  position  and  orientation 
in  the  object  space.  The  position  of  the  photograph  is  defined  by  the  object  space 
coordinates  of  the  perspective  center  XL.  YL.  ZL.  The  orientation  which  describes  the 
attitude  of  the  camera  at  the  moment  of  exposure,  refers  to  the  spatial  relationship 
between  the  object  coordinate  system  and  the  image  coordinate  system.  The  system 
used  here,  and  a  very  common  system,  defines  the  orientation  by  the  orientation  angles 
OMEGA.  PHI.  KAPPA  as  used  in  the  PROJECT  subroutine. 

OMEGA  is  the  first  rotation  around  X.  counterclockwise. 

PHI  is  the  second  rotation  around  Y.  counterclockwise. 

KAPPA  is  the  third  rotation  around  Z.  counterclockwise. 

Thus  the  exterior  orientation  of  the  photograph  is  defined  in  terms  of  XL,  YL, 
ZL,  OMEGA,  PHI,  and  KAPPA. 

7.  Orientation  Matrix 

The  relationship  between  the  image  and  the  object  coordinate  system  is 
expressed  by  a  3  x  3  orthogonal  matrix  where  each  one  of  its  elements  is  a  function  of 
either  one  of  the  angles  OMEGA.  PHI,  KAPPA.  This  matrix  allows  to  transform 
from  the  object  space  system  to  the  image  space  system,  that  is  from  the  real  world  to 
the  photograph,  see  (Fig.  3.2). 

8.  Collinearity  Equations 

Any  individual  pixel  in  the  photograph  can  be  mapped  from  a  3-D  object 
coordinates  by  use  of  the  3  elements  of  interior  orientation  and  6  elements  of  the 
exterior  orientation,  which  describe  the  collinearity  equations.  These  9  elements  are 
constant  for  all  pixel  in  the  image,  but  since  the  photograph  is  a  2-D  representation  of 
a  3-D  object,  the  scale  is  different  from  one  pixel  to  another.  To  avoid  this  problem, 
we  use  the  collinearity  equations  as  shown  in  the  subroutine  PROJECT. 

9.  Photo  to  Digital  Image  Transformation 

Transformation  from  the  photograph  coordinates  to  the  digital  image 
coordinates  is  called  an  'affine'  transformation  and  includes  a  rotation,  two  scale 
changes  and  two  translations.    This  implies  the  solution  of  the  equations: 

XP  =  SX(cos(p)  -  6  sin(p))XI  -(SY  sin(P))YI  +  CI 

YP  =  SX(sin(p)  +  5cos(p))XI  +  (SY  cos(P))YI  +  C2 

where 

XP,  YP    =  Photo  coordinates 
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Figure  3.2     Image  and  object  coordinate  systems. 
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XI,  YI    -  Image  coordinates. 

SX,  SY   =  Scale  changes. 

CI .  C2    =   Translation  in  both  a.xes. 

DELTA      =   Xonperpendicularity  or  affinity  between  the  axes  X,  Y. 

BETA       =   Rotation  angle. 

This  transformation  is  also  called  6-parameter  transformation.  This  process 
has  been  implemented  in  the  subroutine  XY2IJ.  The  following  data  has  been  provided 
for  the  analysis  of  the  aerial  photographs  under  test: 

A I  =  SX{cos($)  -  6"  sin($)) 

.42  =  SY(sin^)  +  6  cos($)) 

Bl  =  -SYsin($) 

B2  =   +SYcos($) 
10.    Data  provided 

(a)  Calibrated  focal  length  for  both  photos  in  millimeters. 

(b)  Calibrated  principal  point  for  both  photos  in  millimeters  (X  and  Y). 

(c)  Radial  lens  distortion  correction  for  both  photos. 

(d)  Disital  image  to  photo  transformations:  the  coefficients  Al,  A2,  Bl,  B2,  Ct, 
C2~in  radians. 

(e)  Exterior  Orientation  Parameters:  OMEGA,  PHI,  KAPPA,  XL,  YL.  ZL.  in 

radians  and  meters  respectively. 

These  concepts  are  implemented  into  the  subroutines  PROJECT.  XY2IJ  and 
AFFIJ,  these  subroutines  are  called  by  the  modules  PTPROJ,  PROJMBR,  PROJEC, 
and  SELREG  with  the  purpose  of  obtaining  the  necessary  interface  between  the 
'geocoordinate'  system  and  the  image  coordinate  system. 
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IV.  THESIS  DEVELOPMENT 

A.  INTRODUCTION 

To  continue  the  study  and  development  of  the  Image  Interpretation  System 
created  by  Prof.  Chin-Hwa  Lee  and  Prof.  Hsi  Jian  Lee,  a  initial  learning  process  was 
necessary.  It  included  Prolog  programming  and  an  Image  Processing  course  to  be 
dictated  by  Prof.  Chin-Hwa  Lee.  as  well  as  general  concepts  in  photogrammetry  and 
some  experience  with  the  use  of  VAX/VMS  system.  FORTRAN  programming  was 
required  to  work  with  existing  programs  beside  using  Prolog.  There  was  also  one 
program  written  in  Pascal. 

The  general  orientation  of  the  thesis  was  to  improve  the  Image  Interpretation 
System  by  reducing  the  CPU  execution  time,  analyzing  memory  usage,  and  detecting 
possible  software  errors  that  could  interfere  with  the  process  of  Change  Detection.  It 
was  suspected  that  some  of  the  nodes  being  analyzed,  were  not  properly  detected  by 
the  Rule  Base.  If  possible,  additional  modelling  capabilities  and  different  approaches  to 
solve  existing  problems  could  be  generated  as  part  of  the  system's  enhancement. 

B.  SOFTWARE  REVIEW 

The  study  of  the  existing  software  was  initiated  with  the  review  of  the  FACTS 
DATABASE  and  the  RULE  BASE.  This  study  facilitated  a  general  understanding  of 
the  problem  solution  approach  as  well  as  providing  a  grasp  of  how  the  'model' 
parameters  interacted. 

The  particular  structure  characteristics  of  the  Prolog  language,  which  include 
'cut'  operators,  variable  'instantiations',  and  a  'backtracking'  behavior,  requires  a  lot  of 
experience  to  analyze  as  in  this  case  a  complex  system.  The  study  of  the  Prolog  part  in 
the  Image  Interpretation  System  was  more  difficult  than  expected,  additionally,  the 
unavailability  of  source  documentation  at  that  moment  made  the  task  of  reading  and 
debugging  more  complicated. 

The  set  of  programs  already  created,  see  (Table  2).  were  developed  in  a  time 
period  of  more  than  a  year  and  half  by  Prof.  Chin-Hwa  Lee  and  Prof.  Hsi  Jian  Lee. 
Because  the  topic  is  complicated,  it  required  several  weeks  to  understand  the  ideas 
behind  every  FORTRAN  program  and  subroutine.  Another  fact  that  complicated  the 
process  of  understanding  was  the  invocation  mechanism  used  by  the  Prolog  Interpreter 
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TABLE  2 
SOFTWARE  STRUCTURE 

IMAGE  INTERPRETATION  SYSTEM 

CHANGE.DAT 

LDES.PRO 

NEHPROP.DAT 

PROJCO.DAT 

PARAfi.DAT 
PROJSHJ.DAT 


r  lags 


I  V  I  '.  J  I  I  D  ^_J  y\ 


Facts   Database 
fact.pl 


Blackboard  storage 
'BOARD' 


■  i  i 
I 


SM 


DEMQN.DAT     Dencnstrat ion  File 


RSAJ1     (Spider  Library) 

PTPPOT    BLKBRD 

i     xjrxvwu      pR0JECT/    XY2IJ 
PROJMBR       PROJECT,   XY2IJ 


M  G  SM 


BLKBRD 
REFINE 


URPROP 

■>  KEASONE 
MEASREG 


PROJEC    ™J£a 

XY2IJ 
npj   npn    BLKBRD, CONVERT 

SHADOW,  AFF1J 

SHDPRO      BLKBRD'    CONVERT 

OUADEQ,    PLUMB,   REGFILL 


ASSREG —  >|jGENTEST 
NEIGHREGS      PUSH,   POP 


CRBKBD 


BLKBRD,    CONVERT 

PRJ1T2,   CQLT1,    STPT2,    AREA2 
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to  initiate  FORTRAN  modules.  This  requires  a  protocol  of  setting  flags  to  indicate 
initiation  and  completion  status.  The  results  of  the  FORTRAN  modules  were  being 
transfered  back,  to  the  Prolog  Interpreter  by  means  of  'mailbox'  interprocess 
communication  process,  or  by  consultation  of  files  generated  by  the  FORTRAN 
modules.  These  files  were  formatted  with  the  required  mnemonics  used  in  the  facts 
database.  The  analysis  of  the  FORTRAN  modules  was  executed  under  schedule,  and 
some  syntax  and  logic  errors  were  corrected.  Concurrent  to  this  analysis,  the  facts 
database  was  corrected  by  Prof.  Chin-Hwa  Lee.  and  the  system  exercised  to  observe 
the  behavior  under  the  new  conditions. 

To  observe  the  results  as  the  process  goes  through  different  stages  in  the 
interpretation  process,  two  files  are  generated.  The  first  is  the  CHANGE.DAT  file, 
which  provides  the  user  with  global  results  derived  from  the  Prolog  Rules.  These  are 
the  certainty  values  used  as  evidence  of  the  existence  of  a  node.  The  file  also  shows 
some  parameters  used  by  the  Change  Detection  Process  as  the  image  file  name  and 
region  description  file  name.  CHANGE.DAT  is  used  as  the  output  file  in  which  the 
Prolog  Interpreter  responds  to  the  entry  command  file  called  RB.COM.  The  last  one  is 
used  as  a  batch  command  file  to  provide  the  system  parameters  and  rule  invocation 
commands. 

The  second  file  generated  to  show  the  results  of  the  process  is  the  file 
DEMON.DAT.  This  is  a  more  detailed  output  that  contains  messages  generated  by 
each  of  the  FORTRAN  subroutines  used  along  the  main  process.  The  information 
written  here  is  used  mainly  for  debug  purposes  because  it  keeps  track  of  the  regions 
modifications  created  by  the  Model  Guided  Split  and  Merge  Module.  Included  in  this 
file  is  information  related  to  coordinates  projected  from  earth  system  3-D  (latitude, 
longitude  and  height)  to  2-D  image  coordinates.  DEMON.DAT  also  presents  the 
assigned  properties  to  the  new  regions  and  the  information  of  the  shadow  area 
generation. 

The  system  can  also  generate  a  set  of  output  files  which  can  provide  an  insight 
into  how  the  process,  as  a  whole  is  carried  out.  These  files  are  generated  when  running 
the  system  under  debug  mode  1.  This  is  a  more  powerful  mode  and  is  used  to 
understand  the  whole  process,  because  it  permits  the  graphic  presentation  of  different 
regions  as  they  are  being  created  by  the  model  guided  split  and  merge  process,  the 
displayed  images  show  the  predicted  shadow  areas  and  projected  model  locations. 
These    files   are   numbered   sequentially    and   have    the   general    form    MSGnnn    and 


27 


DEMnnn,  where  nnn  is  a  sequence  number  beginning  with  0.  These  files  are  used  by 
the  program  HJTEST  to  display  on  the  COMTAL  system  the  labeled  region  map  and 
other  areas  as  indicated  before. 

By  using  these  messages  and  demonstration  files,  is  possible  to  understand  the 
system  behavior  and  detect  possible  errors  in  the  process.  It  is  necessary  to  mention 
here  that  the  subroutine  AFFU  was  created  to  shift  the  'model'  projection  to  its 
expected  position.  Due  to  errors  in  the  collected  data  from  the  'model',  the  projected 
vertices  were  not  coincident  with  the  corners  of  the  buildings  observed  in  the  aerial 
photograph,  some  error  was  being  introduced  in  the  property  results.  The  shadow 
properties  were  not  exactly  coincident  and  a  'affme'  transformation  was  necessary  to 
overlap  the  projected  'model'  with  the  observed  2-D  building  projection  in  the  aerial 
photograph. 

After  the  'affine'  process  was  performed,  a  better  region  matching  was  obtained, 
and  the  shadow  module  produced  improved  results  because  the  projected  predicted 
areas  were  in  position.  However,  5  nodes  were  still  not  providing  enough  evidence  to 
conclude  their  existence.  Further  study  should  be  directed  to  determine  the  cause  of 
this  anomaly. 

Because  the  amount  of  possibilities  that  could  be  presented  to  a  system  of  this 
magnitude  is  enormous,  the  software  review  process  still  continues.  However,  as  will 
be  seen  in  later  sections,  the  system  performance  was  finally  upgraded  and  all  nodes 
were  properly  evaluated,  increasing  with  it  the  percentage  of  system  accuracy. 

The  software  documentation  has  been  written  and  is  included  as  appendices  at 
the  end  of  the  thesis  . 

C.       MEASURING  THE  CPU  TIME 

Another  objective  of  the  thesis  was  the  verification  of  CPU  time  consumption 
while  running  the  Image  Interpretation  System  in  different  debug  modes.  It  has  been 
observed  that  the  system  performs  an  immense  amount  of  calculations,  which  are 
needed  by  the  Image  Processing  operators  in  the  computational  environment  to 
analyze  the  labeled  region  files  and  generate  new  regions  under  the  model  guided  split 
and  merge  module. 

In  order  to  have  an  efficient  Image  Interpretation  System,  the  system  should  be 
able  to  analyze  aerial  photographs  in  reasonable  periods  of  time.  Depending  on  the 
complexity  of  the  buildings  under  consideration,  the  CPU  time  consumption  will  also 
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TABLE  3 
CPU  TIME  CONSUMPTION  (IN  SECONDS) 

MODE-->  DEBUG  0  DEBUG  1 

PROGRAMS 


CRBKBD  8.44  9.07 

>  Node  1  < 


SELREG 
MGSM 

selreg" 

MGSM 

selreg" 

MGSM 

selreg" 

MGSM 

selreg" 

MGSM 

selreg" 

MGSM 

selreg" 

MGSM 

selreg" 

MGSM 

selreg" 

MGSM 

selreg" 

MGSM 

selreg" 

MGSM 

selreg" 

MGSM 


>  Node  2  <  - 

>  Node  3  <- 

>  Node  4  <  - 

>  Node  5  <  - 

>  Node  6  <  - 

>  Node  7  <  - 

>  Node  8  <  - 

>  Node  9  <  - 

>  Node  10  < 

>  Node  11  < 

>  Node  12  < 


3.63 
3.26 

4.32 
6.32 

3. SO 

3.55 

4.39 

23.17 

3.77 
6.24 

4.31 
63.66 

3.65 

5.58 

4.35 
25.57 

3.70 

7.17 

4.41 
79.73 

3.74 
4.36 

4.30 
61.52 

3.73 
5.19 

4.34 
24.27 

3.70 
5.43 

4.39 
25.70 

3.76 
(1) 

4.44 
(1) 

3.68 
2.59 

4.36 
6.63 

3.87 
(1) 

4.52 
(1) 

4.01 
(1) 

4.70 

(1) 

SHDPRO  70.69  93.43 

( 1)  Not  used  module. 


be  different.  For  our  experiment,  we  have  used  a  12S  x  128  image,  and  the  results  are 
listed  in  Table  3  for  a  'model'  composed  of  16  nodes.  From  the  Table  3  it  can  be  seen 
the  great  difference  in  CPU  execution  time  consumption  between  DEBUG  Mode  0  and 
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DEBUG  Mode  1,  which  is  caused  primarily  because  the  last  processing  mode  generates 
a  set  o(  demonstration  images  and  messages  for  everyone  of  the  splited  and  merged 
regions,  it  also  show  in  these  images  the  general  evolution  of  the  segmentation  process 
by  displaying  continuously  the  labeled  region  map. 

Because  there  are  intervals  of  time  between  the  different  process  invocations, 
where  the  CPU  have  not  been  accounted  by  the  program  JCPUT  (  the  CPU  time 
measuring  subroutine  ),  the  total  CPU  time  cannot  be  calculate  precisely.  However,  the 
results  obtained  reflect  reasonable  estimates  when  considering  the  magnitude  of  the 
process. 

D.       INCORPORATING  THE  REFLECTANCE  MODELLING 

1.  Introduction  (Image  Irradiance  and  Surface  Orientation) 

Having  reviewed  the  system  software,  measured  the  CPU  time  performance, 
and  corrected  some  errors  to  improve  system  performance,  there  were  still  some  nodes 
in  the  'model'  that  were  not  being  detected  properly.  Several  factors  were  influencing 
the  evidence  necessary  to  determine  the  existance  of  these  nodes.  One  of  the  most 
critical  factor  was  related  to  the  'certainty  of  tone'.  The  weight  assigned  to  the 
property  combined  with  the  difference  in  the  predicted  and  calculated  tone  was 
lowering  the  final  average  of  certainty  obtained  by  combining  the  certainties  of  size, 
polygon  and  shadow  of  the  node.  Therefore,  the  need  to  obtain  a  more  accurate 
predicted  tone  was  an  obvious  objective  to  be  pursued. 

An  initial  approach  was  suggested  by  the  thesis  Advisor  to  review  some  papers 
including  an  article  on  remote  sensing,  see  (Ref  17),  in  order  to  incorporate  reflectance 
capabilities  to  the  Image  Interpretation  System.  The  task,  was  to  be  carried  out  by  the 
computational  environment  to  perform  the  analysis  and  find  out  the  effects  of  scene 
illumination  over  the  node  surfaces. 

The  problem  of  assigning  a  predicted  tone  for  even.-  node  lies  in  the 
observation  of  surface  normals,  see  (Refs.  18,  19),  sun  position  and  observer  position. 
It  is  necessary  to  find  out  how  they  affect  the  determination  of  an  accurate  predicted 
tone  compared  to  the  calculated  tone  from  the  node  projection  over  the  digitized  image 
and  give  a  reasonable  similarity  measure. 

Tone  synthesis  is  based  on  a  scene  radiance  equation  that  combines  the 
bidirectional  reflectance  distribution  function  BRFD  of  the  surface  material,  see  (Ref. 
17),  the  spatial  and  spectral  distribution  of  light  sources,  and  some  assumptions  about 
the  atmospheric  attenuation  and  path  radiance. 
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2.  Synthesis  of  the  Tone 

Unknown  parameters  of  the  'model'  have  been  obtained  from  the  real  image 
by  sampling  and  experimentation.  The  results  of  comparison  with  the  predicted  tones 
and  the  calculated  tones  from  the  image  will  be  obtained  when  the  certainty  of  tone 
rule  is  executed  by  the  Rule  Base  expert. 

To  synthesize  the  apparent  brightness  of  a  surface,  we  must  consider  the 
intrinsic  reflectance  characteristics  of  the  surface  material,  the  spatial  and  spectral 
distribution  of  the  illumination,  and  the  orientation  of  the  surface  relative  to  the 
observer  and  sun  angle.  The  process  uses  the  knowledge  available  at  the  time  when  the 
photograph  was  taken  to  simulate  the  relative  angle  of  the  light  source.  Also  the 
reflectance  factor  of  the  surface  (ALBEDO),  which  is  unknown,  is  considered  to  be 
ideal  for  the  effect  of  calculations  and  given  a  value  of  1.  We  also  assume  that  the 
surface  materials  forming  the  nodes  are  invariant  with  respect  to  the  rotation  of  the 
surface  normal,  see  (Fig.  4.1). 

To  calculate  the  radiance  equation,  we  need  to  obtain  the  direction  of  the 
incident  angle  I,  the  emergent  angle  E,  and  the  phase  angle  G.  The  incident  angle  I  is 
the  angle  between  the  sun  rays  direction  and  the  normal  to  the  surface.  The  emergent 
angle  E  is  the  angle  between  the  normal  and  the  direction  to  the  observer.  The  phase 
angle  G  is  the  angle  between  the  incident  and  reflected  rays.  For  a  distant  viewer  and 
distant  light  source,  as  in  the  aerial  photograph  case,  the  phase  angle  G  is  constant. 

To  calculate  these  angles,  we  need  to  define  the  gradient  of  the  node  surface 
normal,  and  the  gradient  of  the  sun,  which  is  the  direction  in  the  earth  centered 
'geocoordinate'  system. 

The  BRDF  (Bidirectional  Reflectance  Distribution  Function)  is  an  intrinsic 
property  of  a  surface  material.  It  determines  how  bright  the  surface  will  appear  when 
viewed  from  a  given  direction  and  illuminated  from  another. 

When  illuminating  a  surface  by  a  collimated  source  with  irradiance  EO  from  a 
direction  with  gradient  (PO,  QO)  and  by  considering  the  illumination  from  the  sky  as  a 
hemispherical  uniform  radiance  ES  over  the  visible  hemisphere,  for  the  points  not  in 
shadow  we  get  the  following  equation:  see  (Ref.  17). 

Lr(I,E,G)  =  {E0!PI)*ALBEDO*cos(I)  +  ES*ALBED0*{{1  +  cos{E));2) 

This  implies  the  use  of  the  emergent  angle  E,  because  differing  surface 
elements  'see'  differing  amounts  of  the  light  source  depending  on  surface  slope.  The 
ALBEDO  is  the  reflectance  factor  of  the  surface  considered  ideal  for  the  calculations 
with  a  value  of  1. 
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Source 


Normal 


^Viewer 

The  three  angles  I,  G  and  E  are  used  to  specify  the 
local  geonetry  of  the  incident  and  the  reflected  ray. 


E=I+G 


Figure  4.1     Local  geometry  of  the  incident  and  reflected  rays. 

The  equation  takes  into  consideration  the  gradient  of  the  node  surface  normal, 
and  at  first  approximation,  the  'model'  uses  the  sun  as  a  distant  collimated  source,  and 
the  sky  as  a  hemispherical  uniform  source. 

The  reflectance  modelling  program  was  created,  and  changes  in  the  facts 
database  were  generated  to  accommodate  the  system  to  this  new  capability. 

After  several  tries,  the  parameters  used  in  the  program  were  adapted  to  our 
'model'  and  the  system  performance  was  upgraded.  As  result,  all  the  previous  nodes 
that  were  not  providing  enough  evidence,  now  show  strong  conditions  as  to  be 
evaluated  and  identified  positively  as  existing  and  unchanged  nodes. 
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The  reflectance  modelling  was  performed  with  all  the  different  nodes,  involving 
different  tones  according  to  the  node's  roof  orientation,  surface  material  category  and 
image  conditions.  The  results  showed  correspondence  between  the  calculated  values 
and  the  ones  observed  in  the  aerial  photograph. 

Further  tone  changes  in  the  facts  database  showed  clearly  the  sensitivity  of  the 
system  to  these  changes.  These  experiments  demonstrate  the  accuracy  of  the  Image 
Interpretation  System  and  the  need  of  exercising  the  system  under  different  conditions 
to  calibrate  the  facts  database  to  its  optimum  values. 
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V.  CONCLUSIONS 

Image  Interpretation  of  Aerial  Photographs  is  a  difficult  task  considering  the  vast 
amount  of  calculations  involved  in  the  analysis  of  image  arrays.  By  using  a  computer, 
in  this  specific  case  a  VAX  11,780  running  under  the  VMS  operating  system,  and  using 
an  Expert  System  to  'mimic'  the  human  reasoning  it  has  been  demonstrated  that  the 
process  of  detecting  changes  in  buildings  from  an  aerial  photograph  is  a  reality.  Even 
more,  this  Expert  System,  which  has  been  created  under  a  firm  base,  could  be  able  to 
adapt  to  future  expansions,  or  to  be  implemented  in  other  Prolog  environments. 

The  ideas  behind  the  system  design  are  solid,  promising  and  ambitious,  the 
results  are  reliable  and  effective  within  the  actual  technology  constraints.  The 
objectives  that  were  delineated  in  the  initial  statement  of  the  investigation  have  been 
accomplished  in  the  following  way: 

1.  The  in-depth  analysis  of  all  the  Fortran  programs  and  subroutines  conforming 
the  image  processing  operators  and  the  Prolog  facts  database  and  Rule  Base 
have  been  performed.  Some  logical  and  syntactical  errors  were  found  and 
corrected.  The  documentation  ofboth  environments,  the  Prolog  and  the  image 
processing  programs,  was  generated  and  are  included  as  Appendices  A  and  B. 

2.  The  facts  database  was  modified  to  accept  the  new  reflectance  capabilities 
introduced  to  the  svstem.  The  changes  were  made  according  to  experimental 
data  sampled  from  the  aerial  photographs.  The  Rule  Base  was  changed  in  the 
same  sense,  but  only  minor  changes  were  generated.  These  are  located  in  the 
'corresponding  regio'ns'  rule  and  are  designed  to  receive  the  incoming  results 
from  the  selecting  regions  module,  which  are  ongmallv  created "  by  the 
reflectance  program. 

3.  The  CPU.  time  was  measured,  and  it  showed  a  distributed  consumption  oyer  the 
different  image  processing  operators  and  prolog  area.  The  values  are  within  a 
reasonable  margin  and  didn't  show  any  excessive  time  consumption  in  the 
Image  Interpretation  system. 

4.  The  process  of  exercising  the  system  with  different  kinds  of  photographs  has 
been  partially  simulated  oy  using  a  modified  'model'.  The  results  show  great 
sensitivity  of  the  system  to' changes  in  tonality  and  position  of  the  nodes  in  the 
'model'. 

5.  The  reflectance  modelling  capabilities  were  developed  and  implemented  in  the 
Image  Interpretation  Svstem.  The  overall  results  present  higher  accuracy  in  the 
output  from  the  Rule  Base.  There  is  clear  evidence  that  indicates  the  progress 
obtained  in  the  conclusions  of  matching  the  'model'  to  the  aerial  photograph. 
This  implies  more  confidence  in  the  svstem  productivity. 

The  actual  Image  Interpretation  System  could  be  transformed  into  a  faster,  more 

reliable,  and  flexible  system  by  the  use  of  a  more  powerful  Prolog  compiler  such  as 
QLTNTUS  Prolog.  The  ideas  used  to  develop  this  system  are  well  conceived,  the 
results  are  optimistic  and  the  system  could  be  enhanced  in  the  future  to  generate  its 
own  rules  and  adapt  more  easily  to  a  given  situation. 


APPENDIX  A 
PROLOG  ENVIRONMENT  SUMMARY 

1.        RULE  BASE  EXPERT 

a.  Functions  performed 

Given  a  specific  environmental  'model'  and  a  defined  camera  conditions,  find 
the  changes  in  the  digitized  image  taken  from  the  aerial  photograph. 

b.  Definitions 

The  certainty  of  Node  N  is:  certainty  _childr  en  (N)  *  certainty  ^property  (N), 
where  certainty _childr en  is  the  arithmetic  average  of  the  certainty  values  of  all  children 
of  the  node,  and  certainty  property  is  the  weighted  sum  of  its  properties;  checkinglist  is 
one  of  the  dynamic  databases. 

c.  Program  structure 

The  following  steps  are  accomplished  by  the   process  in  sequence: 

(a)  Get  the  following  input  parameters  from  the  user  (example  data): 

Labeled  region  file:        Labfile  (LLAB.DAT). 

Image  file:  Imgfile  (L220200.DAT). 

Side  of  the  image:  Side  as  (LEFT). 

Debug  Mode:  Debug  (0/1/2). 

Image  Identification:     Imageld  (LEFT_PICT1). 

Prolog  region  description  file:   Desfile  as  LDES.PRO. 

(b)  Write  the  parameters  in  the  file  PARAM.DAT  as  well  as  the  affine  coefficientes 
for  the  respective  side  of  the  stereo  pair  of  images. 

(c)  Create  the  blackboard  interprocess  communication  area  by  initiating  the  fortran 
program  CRBKBD  with  the  statement:  'Simage'  (crbkbd,  {},  X  ). 

(d)  The  Prolog  expert  must  consult  the  region  description  file  to  get  all  the  region 
properties.  Continue  reading  the  entry  command  file  RB.COM  and  proceed  with 
the  subsequent  control  rules. 

d.  Called  routines 

CRBKBD:  Creation  of  the  'blackboard'  for  interprocess  communication. 

e.  Other  considerations 
None. 
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2.        FACTS  DATABASE 

a.  Functions  performed 

The  prolog  file  FACT. PL  serves  as  the  environmental  'model';  it  has  a  'tree' 
like  structure  and  it  is  the  static  database  of  the  Change  Detection  System,  its  tree 
structure  is  composed  of  a  number  of  nodes,  and  each  node  has  a  unique  name  with 
almost  similar  format.   The  environmental  'model'  has  the  following  characteristics: 

(a)  The  hierarchical  relationship  in  the  tree  is  fixed;  the  hierarchy  from  top  to 
bottom  of  the  building  family  is  as  follows:  block,  building  cluster,  building, 
building  parts  and  subparts. 

(b)  Property  inheritance:  By  this  property,  the  system  extracts  the  missing  properties 
of  a  node  from  the  ones  of  its  ancestors. 

b.  Definitions  as  used  in  the  Prolog  expert 
coefficient:  Affine  transform  coefficients. 
smc_tone:  Surface  Material  Category. 

shadow  smc  tone:    Variation  of  smc  tone  used  for  shadow  areas. 


block: 

type: 

class: 

children: 

from: 

mbr: 


A  collection  of  tree  roots. 

Defines  the  classification  of  hierarchies. 

Describes  the  type  of  environmental  'model'. 

Are  the  roots  originated  in  the  node. 

Relates  a  node  to  its  ancestors. 

Is  the  Minimum  Bounding  Rectangular  area  enclosing  a  node. 


background_smc:    The  background  smc  surrounding  the  node  base  plane. 


context: 

compact: 

no_vertex: 

polygon: 

height: 

property: 

ref_point: 

predicted_area: 


Gives  a  written  statement  about  the  node  identification. 

Gives  a  simple  meaning  of  the  general  shape  of  the  node. 

The  number  of  vertices  in  the  node  polygon. 

Describes  the  3-D  coordinates  of  each  vertex  in  the  node. 

Provides  the  node's  height. 

Gives  the  properties  and  weights  that  should  be  analized  for 

a  given  node. 

Reference  point  used  to  observe  the  sensed  image  light 

conditions. 

Opposite  corner  coordinates  of  the  the  rectangle  enclosing 

the  block  used  to  detect  if  the  sensed  image  holds  the  area 

of  interest  for  the  change  detection  process. 
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c.  Called  routines 

None. 

d.  Other  considerations 

None. 

3.  CONTROL  RULE  #1 

a.  Functions  performed 

Find  the  reference  point  ratio  in  order  to  modify  the  calculations  of  the 
change  detection  process  accordingly  to  the  light  conditions  on  the  image.  Use  the 
find _ref_r alio  rule. 

b.  Definitions 
None. 

c.  Program  structure 

The  following  steps  will  be  accomplished  by  the   process: 

(a)  Get  from  the  facts  database  the  reference  point  coordinates.  Obtain  the  2-D 
projection  coordinates  of  the  reference  point;  use  the  average  value  of  the 
surrounding  pixels  in  the  image  to  get  a  more  accurate  reference. 

(b)  Compare  to  the  facts  database  reference  value  and  establish  a  ratio  that  will  be 
used  through  the  whole  change  detection  process. 

d.  Called  routines 

The  program  PTPROJ  is  initiated  with  the  sentence:  'Simage'  (  ptproj,  Coord, 
List). 

e.  Other  considerations 
None. 

4.  CONTROL  RULE  #2 

a.  Functions  performed 

Mark,  a  block  as  a  root  node  for  subsequent  change  detection  process  if  the 
intersection  area  of  it  and  a  segmented  image  is  large  enough. 

b.  Definitions 
None. 

c.  Program  structure 

The  following  steps  will  be  accomplished  by  the   process: 
(a)      Select  a  block  from  the  facts  database,  use  type  (X, block),  and  get  its  MBR  before 
initiating   the  fortran  program  PROJMBR;   once  we   get   back   the  2-D   image 
projection,  use  the  SIDE  parameter  to  extract  the  X  and  Y  coordinates  of  the 

37 


projection  of  the  block,  for  the  left  or  right  side  of  the  image  stereo  pairs  use  the 
routine jprojectjnbr  and  separate  jprojectjnbr  rules. 

(b)  Obtain  the  image  file  identification  and  use  it  to  extract  the  predicted  area  of  the 
block  from  the  facts  database.  Use  the  coordinates  of  the  predicted  area  and  the 
ones  provided  by  the  PROJMBR  program  to  obtain  the  area  of  intersection  of 
both  MBRs;  use  the  inter section jbetwjnbrs  rule. 

(c)  Get  the  size  of  the  projected  mbr  and  the  size  of  the  intersection  part,  use  the 
mbrjsize  rule;  obtain  the  ratio  by  dividing  the  'intersection  MBR'  size  over  the 
'projected  MBR'  size:  use  the  ratio  (Size_Ints,  Size_MBR2,  R)  rule. 

(d)  Check  the  ratio  of  intersection:  if  the  block  is  identified  to  be  in  the  image,  its 
ratio  should  be  higher  and  the  change  detection  process  could  be  initiated;  use 
the  check_block  rule;  the  ratio  must  be  over  60  per  cent  in  order  to  determine 
that  the  block  is  positively  located  the  image. 

(e)  By  the  same  ideas,  locate  the  building  cluster  from  the  block,  get  the  building 
from  the  cluster  and  use  the  ratio  of  MBR's  size  intersection  before  checking  the 
building  in  order  to  be  sure  that  is  properly  located  in  the  sensed  image:  when  the 
building  terminal  nodes  are  detected,  add  them  to  the  checking  list,  which  will  be 
constructed  as  a  list  of  all  the  elements  to  be  analized  along  the  change  detection 
process.  Use  the  following  rules:  find ' _building_cluster  jnter section, 
find  jnter  section  jijbuilding,  checkjbuilding  and  putjiode  to  locate  and  add  the 
node  to  the  checking  list. 

d.  Called  routines 

PROJMBR:  initiated  by  using  the  statement  'Simage'  (project_mbr.   MBR. 
Proj_MBR). 

e.  Other  considerations 
None. 

5.        CONTROL  RULE  #3 

a.  Functions  performed 

Activate  one  of  the  nodes  in  the  'checkinglist'  dynamic  database. 

b.  Definitions 
None. 
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c.  Program  structure 

The  following  steps  will  be  accomplished  by  the   process: 

(a)  Check  if  it  is  the  last  node  in  the  'checkinglist'(  the  last  one  must  be  equal  to 
"markL"  )  by  using  the  rule  last jiode jest  {Node). 

(b)  If  it  is  not  the  last  node,  validate  it  through  the  analysis  process;  use  the  control 
rule  4:    validate(Node)  and  retract  it  from  the  'checkinglist'  when  finished. 

d.  Called  routines 
None. 

e.  Other  considerations 
None. 

6.  CONTROL  RULE  #4 

a.  Functions  performed 

If  the  node  is  a  non-terminal  node,  then  add  its  children  to  the  'checkinglist'. 

b.  Definitions 
None. 

c.  Program  structure 

The  following  steps  will  be  accomplished  by  the   process: 

(a)  Get  the  children  list  of  the  node  from  the  facts  database  and  use  the  putjiode 
rule  to  initialize  its  certainty  value  to  -9999. 

(b)  Include  the  node's  children  in  the  'checkinglist';  use  the  validate  {Node)  rule. 

d.  Called  routines 
None. 

e.  Other  considerations 
None. 

7.  CONTROL  RULE  #5 

a.  Functions  performed 

If  node  is  a  terminal  node,  and  its  certainty  value  is  undecided,  then  calculate 
its  certainty  value  by  combining  all  the  property's  values  obtained;  update  its  father 
certainty  and  eliminate  it  from  the  checkinglist. 

b.  Definitions 
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c.  Program  structure 

The  following  steps  will  be  accomplished  by  the   process: 

(a)  Calculate  the  property  certainty  of  the  node  by  first  initializing  the  property 
certainty  value  to  -9999;  get  the  list  of  properties  and  its  weights  for  the  specified 
node  from  the  facts  database;  obtain  the  combined  certainty  value  for  all  the 
properties,  by  multiplying  the  weights  of  the  individual  properties  by  its  certainty 
factors  and  dividing  the  result  over  the  sum  of  all  the  weights,  use  the 
validate(Node)  rule. 

(b)  Report  its  status  (see  control  rule  10). 

(c)  Refine  the  results  by  the  'model'  guided  split  and  merge  program;  if  the  results 
are  not  satisfactory  (see  rule   No.  1 1 ). 

(d)  Update  the  father  certainty  of  the  node  (see  rule  No.  8). 

(e)  Remove  the  node  from  the  'checkinglist'  (see  ruleNo.  6). 

d.  Called  routines 
None. 

e.  Other  considerations 
None. 

8.        CONTROL  RULE  #8 

a.  Functions  performed 

If  the  certainty  values  of  the  node's  children  are  completely  decided,  then 
update  the  certainty  value  of  the  node's  father. 

b.  Definitions 
None. 

c.  Program  structure 

The  following  steps  will  be  accomplished  by  the   process: 

(a)  Obtain  from  the  facts  database  the  list  of  all  children  corresponding  to  a  node's 
father,  use  the  from  (Node,  Father)  and  children  (  Father,  L)  facts. 

(b)  Get  all  the  children's  certainty  value;  if  one  of  them  have  not  been  solved  yet, 
return;  use  the  all_children_solved  (  L,  ChildrenNo)  rule. 

(c)  Combine  the  children  certainty  factors  and  get  the  average  by  dividing  the  total 
sum  of  children's  certainty  over  the  number  of  children,  use  the 
combine _certaintyjof_childr en  (  L,  Children_No,  XI)  rule. 
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(d)  Find  the  father's  property  certainty;  in  case  that  there  are  particular  properties  to 
find,  like  'shadow',  use  the  find^fatherjpropertyjzertainty  (Father,  Value)  and 
calculate  j?roperty_certainty  (Father)  rules,  as  seen  in  control  rule  No. 5  part  (a). 

(e)  Combine  the  children  and  father  certainties;  if  one  of  them  is  equal  to  -9999  or  0, 
use  the  other  to  obtain  the  combined  value,  use  the 
combine  _children_and_property_cer[ainty  rule. 

(0  Replace  the  'instantiated'  value  of -9999,  report  its  status  and  repeat  the  process 
for  the  upper  hierarchies. 

d.  Called  routines 
None. 

e.  Other  considerations 
None. 

9.  CONTROL  RULE  #9 

a.  Functions  performed 

If  the  certainty  value  of  a  node  is  decided,  then  remove  the  node  and  its 
successors  from  the  'checkinglist'. 

b.  Definitions 
None. 

c.  Program  structure 

The  following  steps  will  be  accomplished  by  the   process: 
(a)      Get  the  node's  children  list  and  remove  the  list  from  the  facts  database;  use  the 
remove _from_checkinglist  and  remove  _childr  en  rules. 

d.  Called  routines 
None. 

e.  Other  considerations 
None. 

10.  CONTROL  RULE  #10 

a.  Functions  performed 

If  the  certainty  value  of  a  node  is  decided,  then  report  its  status. 

b.  Definitions 
None. 
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c.  Program  structure 

The  following  steps  will  be  accomplished  by  the  process: 

(a)  Get  the  node's  certainty  value  from  the  facts  database  using  the  certainty _yalue  ( 
Node,  Value)  fact;  if  the  value  is  -9999  return. 

(b)  Check  the  value  against  certain  limits  to  decide  different  conclusions;  write  the 
results  in  the  CHANGE.DAT  report  file,  use  the  status  {Node,  Value)  rule. 

d.  Called  routines 
None. 

e.  Other  considerations 
None. 

11.      CONTROL  RULE  #11 

a.  Functions  performed 

Split  and  merge:  If  the  certainty  of  a  node  cannot  be  decided  after  all 
properties  are  evaluated,  then  try  to  split  and  merge  the  regions  associated  with  the 
node  (these  are  the  corresponding  regions  obtained  by  the  correspondingjregions  rule). 

b.  Definitions 
None. 

c.  Program  structure 

The  following  steps  will  be  accomplished  by  the  process: 

(a)  If  the  certainty  value  of  the  node  is  already  greater  than  60  per  cent,  return. 

(b)  If  not,  get  the  possible_regions  list  that  was  generated  by  the  corresponding 
regions  rule  and  call  the  MGSM  program  to  reconfigure  the  regions  intersecting 
the  node's  projection,  in  order  to  evaluate  more  precisely  the  different  property 
certainty  factors,  use  the  possible jregions  (  Node,  RegList)  fact. 

(c)  The  facts  database  is  modified  according  to  the  results  produced  by  the  MGSM 
program,  these  results  include  the  new  region  properties  in  the  file 
NEWPROP.DAT,  use  the  update jxjrecord  (  Term)  rule. 

(d)  Calculate  the  property  certainties  as  was  done  in  the  control  rule  No.  5  and 
report  the  new  status,  use  the  calculate jproperty ^certainty  (  Node)  rule. 

d.  Called  routines 

MGSM:  initiated  by  the  sentence  'Simage'  (mgsm,  RegList,  X). 
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e.   Other  considerations 

None. 

12.  CERTAINTY  OF  TONE 

a.  Functions  performed 

The  certainty  of  tone  is  assigned  according  to  the    average  value  of  the  tone 
difference  of  all  the  corresponding  regions. 

b.  Definitions 
None. 

c.  Program  structure 

The  following  steps  will  be  accomplished  by  the  process: 

(a)  Get  the  list  of  corresponding  regions  to  the  node's  projection;  use  the 
corresponding _regions  rule. 

(b)  Get  the  surface  material  category'  of  the  node  from  the  facts  database  and 
multiply  it  by  the  ratio  obtained  in  control  rule  No.  1,  this  is  the  node  relative 
gray  level. 

(c)  Obtain  the  difference  of  tone  in  each  one  of  the  corresponding  regions,  with  the 
relative  gray  level  of  the  node  being  analized.  Get  the  list  of  tone  differences  and 
sizes  of  each  tone  included  in  the  corresponding  regions  list,  use  the  subjonejiiff 
and  Ust_ofjone_diffxVi\cs. 

(d)  Obtain  the  weighted  average  by  multiplying  and  summing  the  tone  and  size  of 
each  one  of  the  corresponding  regions  over  the  sum  of  the  intersected  region's 
area,  use  the  weighted _sum  rule. 

(e)  Assign  a  tone  certainty  value  according  to  the  weighted  average  obtained,  use  the 
assign_value_tone  rule. 

d.  Called  routines 
None. 

e.  Other  considerations 
None. 

13.  CERTAINTY  OF  SIZE 

a.  Functions  performed 

Assign  a  certainty  value  according  to  the  ratio  between  the  node  size  and  the 
sum  of  all  the  corresponding  region's  sizes  intersecting  the  projected  node. 
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b.  Definitions 

None. 

c.  Program  structure 

The  following  steps  will  be  accomplished  by  the  process: 

(a)  Get  the  list  of  the  corresponding  regions  and  their  sizes,  get  the  mapped  size  of 
the  node  from  the  facts  database  generated  by  the  corresponding_regions  rule. 

(b)  Assign  a  value  according  to  the  ratio  between  the  mapped  size  and  the  sum  of 
the  list,  use  the  RATIO  fact  and  assign  _value_size  rule,  to  generate  the  certainty 
value  of  this  rule. 

d.  Called  routines 
None. 

e.  Other  considerations 

The  list_of_region_size  rule  gets  from  the  facts  database  the  sizes  of  all  the 
corresponding  regions.  The  RATIO  is  defined  as  mapped  node  size  over  sum  of  the 
corresponding  regions;  the  last  predicate  is  obtained  through  the  sumjist  rule. 

14.      CERTAINTY  OF  SHADOW 

a.  Functions  performed 

Assign  a  value  to  the  shadow  certainty  according  to  the  predicted  and 
calculated  shadow  areas  projected  from  the  node. 

b.  Definitions 

The  property  inheritance:  a  concept  used  when  the  property  of  a  node  is  not 
found,  it  is  searched  through  its  ancestors;  to  obtain  it  use  the  find _backgr  ound _smc 
rule. 

c.  Program  structure 

The  following  steps  will  be  accomplished  by  the  process: 

(a)  Obtain  the  node's  descendants:  get  the  node's  children  from  the  facts  database. 
and  with  this  information,  create  a  table  of  possible  descendants  initializing  the 
last  element  to  "markL"  ;  continue  searching  for  new  generations  until  all 
descendants  are  collected;  use  the  find _ne. regeneration,  collect jiescendant  and 
getnext_des  {DES)  rules. 

(b)  Generate  the  shadow  projected  polygons:  for  even'  descendant  node  get  the 
number  of  vertices  forming  the  polygon  and  extract  its  coordinates  to  be  written 
into    the    file   PROJSH.DAT;   use   the  form_proj_polygons   and   write  _jnoj_coord 
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rules.  Note  that  the  projected  shadow  coordinates  were  originally  generated  by 
the  'corresponding  regions'  rule,  which  called  the  SELREG  program,  and  wrote 
them  in  the  PROJCO.DAT  tile. 

(c)  Initiate  the  shadow  processing  program  SHDPRO  with  the  statement  'Simage' 
(shdpro,  (},  L). 

(d)  After  the  program  SHDPRO  generates  a  list  of  area-tone  pairs,  these  are  the 
different  regions  intersecting  the  projected  shadow  areas;  obtain  the  average 
wighted  tone  value  after  summing  the  weighted  tones  and  dividing  proportionally 
to  their  areas:  use  the  weighted _sum_shadow  rule. 

(e)  Assign  a  value  to  the  shadow  certainty,  according  to  the  absolute  difference 
between  the  predicted  tone  of  the  projected  shadow,  and  the  calculated  tone  of 
the  sum  of  the  intersecting  regions  taken  from  the  image  file;  use  the 
assign  _value_shadow  and  certainty _value  shadow  rules. 

d.  Called  routines 

SHDPRO:  Shadow  processing  program. 

e.  Other  considerations 
None. 

15.      CERTAINTY  OF  POLYGON 

a.  Functions  performed 

Generates  a  certainty  value  according  to  the  ratio  between  the  intersecting 
parts  of  the  regions  overlapping  a  node  and  the  mapped  size  of  it. 

b.  Definitions 
None. 

c.  Program  structure 

The  following  steps  will  be  accomplished  by  the  process: 
(a)  Get  the  mapped  size  of  the  node  from  the  facts  database;  obtain  also  the 
intersected  area's  size  of  the  different  regions  by  using  the  mapped_size  and 
intersect_area  facts:  add  the  list  of  intersection  areas  and  obtain  the  ratio  by 
dividing  it  over  the  mapped  size;  use  the  sumjist  rule;  assign  a  certainty  value 
according  to  the  ratio;  use  the  assign _value _polygon  rule. 

d.  Called  routines 
None. 
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e.   Other  considerations 

None. 

16.      PROLOG  COMMON  ROUTINES 

a.  Functions  performed 

Provide  information  for  several  properties  and  rules  through  the  change 
detection  process. 

b.  Corresponding  regions  rule: 

The  purpose  of  this  rule  is  to  find  the  regions  related  with  a  specific  node;  it 
produces  the  following  facts  and  tables  to  be  used  by  the  Interpreter:  mapped  size  of 
the  node,  list  of  regions  corresponding  to  a  node  area  and  areas  of  intersection,  also 
generates  through  program  SELREG,  and  the  projected  and  shadow  predicted 
coordinates  of  the  node  vertices.  The  following  steps  will  be  accomplished  by  the 
process: 

(a)  Check  if  the  possible  regions  table  already  exists:  return  if  so.  If  not  create  a 
vertex  file  called  VERTEX.DAT  to  be  used  by  program  SELREG  call  it  by  the 
command  'Simage'  (selreg,  (},  VI). 

(b)  From  the  list  received  through  mailbox  created  at  the  conclusion  of  SELREG. 
generate  the  list  of  intersecting  areas  and  possible  regions  overlapping  the 
projected  node;  use  the  collect  ^regions  and  region_nwnber_and_intersec_size  rules. 

c.  Mathematical  common  routines. 

The  mathematical  common  routines  are  used  to  find  differences,  rnin,  max  and 
absolute  values. 

d.  Utility  common  routines 

Are  used  to  find  the  minimum  number  in  a  list  as  minjist,  and  number  of 
elements  in  a  list  as  no_elements  rule. 

e.  Called  routines 
None. 

f.  Other  considerations 
None. 
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APPENDIX  B 
COMPUTATIONAL  ENVIRONMENT  SUMMARY. 

1.        PROGRAM  (FORTRAN)  'SELREG' 

a.  Functions  performed 

Select  a  list  of  regions  intersecting  with  the  area  projected  by  a  node  from  the 
environmental  'model'  and  get  a  list  of  the  image  projection  coordinates  in  2-D 
corresponding  to  the  vertices  of  the  node  and  their  shadows.  This  information  will  be 
written  into  a  file  to  be  used  by  the  shadow  verification  subroutine.  Additionally,  this 
program  will  handle  the  modification  of  the  facts  database  according  to  new  tone 
values  generated  by  to  the  sun  position  relative  to  the  node  surface  normal. 

b.  Input 

NODENAME:       The  node  being  analized. 

ITONE:  The  node  tone  before  sun  illumination  analysis. 

COORD:  The  earth  coordinates  X.Y.Z  of  every  node  vertex. 

IN_ARGS:  The  number  of  arguments  received  through  mailbox. 

NOVERTEX:         The  number  of  vertices  for  each  node. 

c.  Output 

VERTEX:  The  2-dimension  coordinates  of  each  vertex  projection. 

IS,  IJ:  The  2-dimension  coordinates  of  each  vertex  shadow  projection. 

ITONE:  The  modified  value  of  the  tone  for  each  node. 

NOREG:  The  region  intersecting  the  node  projection. 

NOPOINTS:  The  number  of  pixels  related  to  the  intersection  area. 

OBUF:  The  output  buffer  to  be  sent  through  mailbox. 

NARGS_OUT:       The  number  of  arguments  being  send  out. 

d.  Common  area 

The  common  area  called  BOARD  has  been  developed  to  provide  interprocess 
communication  through  a  background  storage  and  includes: 

MODEL:  This  matrix  provides  the  'model'  2-D  projection. 

REGMAP:  Includes  all  the  segmented  regions. 

LR:  Side  of  the  stereo  pair  of  images. 

SL'NANG:  Provides  with  the  absolute  sun  direction. 

PLNCOEF:  Are  the  base  plane  coefficients  A?  B.  C  and  D. 

FILESEQ:  A  value  used  to  identify  the  progress  report  files. 
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COEF: 
NREGS: 

AREA: 
PERI:     , 
COMPACT: 
MTONE: 
VTONE: 
MBR: 
DEBUG: 
IMAGE: 
e.  Work  area 
NCHARS: 
MBR2D: 
STRBUF: 
INUM: 
U.  V.  W: 
COL'NTX.l: 
ROOF: 
NO_REGS: 
RATIO: 
NO_MODEL: 
MBRINT: 


Coordinates  corresponding  to  the  exposure  station. 

Is  the  number  of  regions  identified  in  the  photograph. 

The  region's  area. 

The  region's  perimeter. 

Parameter  that  show  a  generated  shape  relation  of  the  region. 

The  average  tone  of  each  region. 

The  tone  variance  of  even7  region. 

The  minimum  boundary  rectangle  for  even.'  region. 

The  type  of  debug  process  under  execution  (0.1  or  2). 

The  digitized  image  of  the  aerial  photograph. 

The  number  of  characters  in  the  node  name. 

The  MBR  of  every  node  projection. 

A  buffer  to  hold  the  input  data  for  the  program. 

Number  of  elements   returned  by  subroutine  CONVERT. 

Real  values  for  X.  Y  and  Z. 

Variables  to  hold  the  CPU  time  consumed  by  the  program. 

Matrix  to  hold  the  real  values  of  the  node's  vertices. 

The  number  of  regions  intersecting  the  node  projection. 

The  relation  of  the  intersected  part  to  the  'model'  size. 

Number  of  pixels  in  the  'model'. 

The  MBR  of  the  intersection  part  for  every  corresponding 


f.  Parameters  and  modes 

This  program  is  designed  to  work  with  debug  modes  0  and  1  (mode  2  is  under 
development),  when  running  in  mode  1  there  is  additional  information  generated  which 
will  be  displayed  in  the  COMTAL  environment  to  show  the  process  progress  report. 
The  program  uses  as  parameters,  ISX  and  ISY  with  values  of  128  which  are  the 
ranges  in  X  and  Y  of  the  aerial  photograph;  MAXREG  is  100  and  it  is  the  limit  of 
regions  to  be  obtained  from  the  segmented  image:  finally  the  left  side  of  the  stereo  pair 
is  assigned  as  a  value  of  1  and  the  right  one  as  2. 

g.  File  handling 

DEMON.DAT:       Output  file  that  contains  all  process  messages. 
VERTEX.DAT:      Input  file  created  by  the  change  detection  process 
in  order  to  provide  with  the  input   parameters. 
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PROJCO.DAT:      Output  file  to  be  consulted  by  the  shadow  processing 

subroutine,  it  include  projected  and  shadow  coordinates, 
h.  Program  structure 

The  program  for  region  selection  is  summarized  in  the  following  steps: 

(a)  The  selecting  regions  program  is  initiated  by  the  Prolog  Interpreter,  and  the  first 
task  is  to  establish  communication  with  the  memory  management  interprocess 
routines:  event  flag  association,  mailbox  communication  and  logical  unit 
association. 

(b)  The  file  DEMON.DAT  is  opened  to  receive  all  the  information  regarding  the 
development  of  the  process  and  the  file  VERTEX.DAT  is  opened  to  provide  all 
the  input  parameters;  next  all  the  work  area  variables  are  initialized. 

(c)  Immediately  for  every  vertex  are  calculated  its  2-D  projected  and  shadow 
coordinates  and  also  generated  the  minimum  boundary  rectangular  of  the  node. 
The  new  tone  for  the  node  is  then  calculated  with  all  the  information  relating  the 
plane  equations  of  the  different  node  vertex  combinations. 

(d)  The  MODEL  matrix  is  progressively  being  generated  with  all  the  accumulated 
2-D  projection  of  the  different  nodes,  where  the  previous  processed  nodes  are 
represented  by  code  '2'  and  the  last  node  by  T';the  last  node  being  processed  is 
retained  in  the  matrix  BUF2D  to  be  used  for  intersection  calculations. 

(e)  Finally  all  the  intersecting  regions  and  their  intersection  area  will  be  transfered 
back  to  the  calling  program  via  mailbox  interprocess  communication. 

i.    Exception  handling 
Not  used, 
j.       Calling  routines 

This  program  is  initiated  by  the  Prolog  Interpreter  during  execution  of  the 
statement  "'Simage'  (selreg,{},M)"  in  the  "corresponding-regions"  rule, 
k.      Called  routines 

JCPUT:  Used  to  calculate  the  CPU  time. 

BLCKBRD:  To  obtain  connection  with  the  blackboard  storage. 

AFFIJ:  Generate  additional  affine  transformation  to  the  image. 

CONVERT:  To  transform  the  ascii  to  integer  values. 

PROJECT:  To  project  a  3-D  coordinate  to  2-D  coordinate. 

XY2IJ:  Perform  the  affine  transformations  of  the  image. 

SHADOW:  Used  to  generate  the  shadow  projections. 

REFLEC:  To  calculate  the  tone  variation  of  the  node. 
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REGFILL:  To  fill  the  intersected  part  of  a  region  and  node. 

FILENAM:  Used  to  provide  a  different  filename  for  the  progress 

report  process  while  working  in  mode  1. 
1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 

(a)  In  order  to  link  this  program,  the  following  command  will  have  to  be  given: 
Slink  selreg.  affij,  blkbrd,  shadow,  convert,  ufo_create.  ufo_open,  filenam,  jcput, 
reflec.  projmbr.  projaff.  regfill,  blkbrd.  opt. 

(b)  The  changes  performed  to  the  program  are  related  to  the  insertion  of  the  timing 
routines,  and  with  the  generation  of  the  new  tone  for  the  different  nodes 
according  to  the  sun  position. 

2.        PROGRAM  (FORTRAN)  'MGSM' 

a.  Functions  performed 

Is  the  mam  program  of  the  'model'  guided  split  and  merge  module. 

b.  Input 

IX_ARGS:  The  number  of  arguments  received  through  mailbox. 

REGLIST:  List  generated  by  corresponding  regions  rule. 

c.  Output 

The    output    of  this    program   is   transfered   to    the    Prolog    Expert   through 
mailbox. 

d.  Common  area 

The  common  area  called  BOARD  has  been  developed  to  provide  interprocess 
communication  through  a  background  storage  and  includes: 

MODEL:  This  matrix  provides  the  'model'  2-D  projection. 

REGMAP:  Includes  all  the  segmented  regions. 

LR:  Side  of  the  stereo  pair  of  images. 

SUNANG:  Provides  with  the  absolute  sun  direction. 

PLNCOEF:  Are  the  base  plane  coefficients  A,  B,  C  and  D. 

FILESEQ:  A  value  used  to  identify  the  progress  report  files. 

COEF:  Coordinates  corresponding  to  the  exposure  station. 

NREGS:  Is  the  number  of  regions  identified  in  the  photograph. 

AREA:  The  region's  area. 

PERI:  The  region's  perimeter. 


50 


COMPACT:  Parameter  that  show  a  generated  shape  relation  of  the  region. 

MTONE:  The  average  tone  of  each  region. 

VTONE:  The  tone  variance  of  even'  region. 

MBR:  The  minimum  boundary  rectangle  for  every  region. 

DEBUG:  The  type  of  debug  process  under  execution  (0,1  or  2). 

IMAGE:  The  digitized  image  of  the  aerial  photograph. 

e.  Work  area 
Not  used 

f.  Parameters  and  modes 

This  program  is  designed  to  work  with  debug  modes  0  and  1  (mode  2  is  under 
development),  when  running  in  mode  1  there  is  additional  information  generated  which 
will  be  displayed  in  the  COMTAL  environment  to  show  the  process  progress  report. 
The  program  uses  as  parameters,  ISX  and  ISY  with  values  of  128  which  are  the 
ranges  in  X  and  Y  of  the  aerial  photograph;  MAXREG  is  100  and  it  is  the  limit  of 
regions  to  be  obtained  from  the  segmented  image;  finally  the  left  side  of  the  stereo  pair 
is  assigned  as  a  value  of  1  and  the  right  one  as  2. 

g.  File  handling 

DEMON.DAT:      Output  file  that  contains  all  process  messages, 
h.  Program  structure 

The  program  for  split  and  merge  is  summarized  in  the  following  steps: 
(a)      This  is  the  main  program  of  the  module,  is  in  charge  of  handling  the  interprocess 
communication    and    passing    parameters    to    the    subroutine    REFINE,    which 
practically  performs  all  the  splitting  and  merging  tasks, 
i.    Exception  handling 
Not  used, 
j.       Calling  routines 

This  program  is  initiated  by  the  Prolog  Interpreter  during  execution  of  the 
statement  "'Simage'  (mgsm,{},M)"  in  the  control  rule  No.  11. 
k.      Called  routines 

REFINE:  To  refine  the  region's  association  with  the  node. 

1.      Timing  constraints 
Not  observed. 
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m.    Other  considerations 

(a)  In  order  to  link  this  program,  the  following  command  will  have  to  be  given: 
Slink  mgsm,  refine,  assreg.  neighreg.  tone,  measone.  measreg,  wrprop.  jcput, 
lilenam.  ufo_create.  ufo_open.  blkbrd,  npsSlibrary:spiderlib  lib 

3.        PROGRAM  (FORTRAN)  'SM' 

a.  Functions  performed 

To  segment  an  image  based  on  an  split  and  merge  algorithm.    The  program 
uses  the  RSAM  segmentation  process  which  is  one  of  the  SPIDER  library  subroutines. 

b.  Input 

Input  data  is  entered  interactively  by  the  user. 

c.  Output 

A  tone  data  file,  and  a  region  labeled  file  are  generated. 

d.  Common  area 
Not  used. 

e.  Work  area 
FNAME: 
OUTFILE: 
INBUF: 
IP: 

MAP: 
LO: 
LN: 
JERR: 
JP: 
NP: 
ITHV: 


The  image  file  name  as  L220200.DAT. 

The  output  labeled  region  file  as  LLAB.DAT. 

Input  buffer  to  hold  each  input  file  record. 

The  image  data  in  integer  format,  values  from  0  to  255. 

Character  codes  to  label  the  segmented  regions. 

Level  at  which  segmentation  starts  (default  =  3). 

Logarithm  to  base  2  of  ISX  and  ISY  =  7  (for  ISX=  128). 

Return  error  code. 

Output  labeled  buffer. 

Index  of  labeling  codes  used  with  "MAP". 

Maximum  difference  for  merging  and  splitting  a  region. 

the  default  value  is  20  (over  256  possible  max). 
NTAREA:  Area  threshold  for  small  region  elimination  performed 

after  each  region  grouping  (default  =  ISX*ISY  200). 
NTINT(ntarea):      One  dimensional  array  in  which  threshold  of  difference 
in  image  intensities  is  stored  as  function  of  area  in 
small  region  elimination  (def.  values  are  50  to  30). 
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f.  Parameters  and  modes 

A  noise  region  is  defined  as  ISX*ISY/1000. 

g.  File  handling 

FN  A  ME:  Input  file  with  the  digitized  image  data. 

OUTFILE:  Output  file  with  the  labeled  regions. 

TONE. DAT:  Output  file  with  the  tones  of  all  labeled  regions, 

h.  Program  structure 

The  following  steps  will  be  accomplished  by  the  segmentation  process: 

(a)  Get  the  input  parameters  :  The  image  file  name,  output  labeled  region  file  and 
prepare  them  to  process  the  input/output  data,  also  the  file  TONE. DAT  is  used 
to  place  all  the  tone  information  related  to  the  different  regions. 

(b)  For  diagnostic  purposes,  the  process  display  the  multiples  of  16  No.s  of  records 
as  they  are  being  read  from  the  input  file;  load  the  image  data  after  transforming 
it  from  byte  format  to  integer  format  with  values  varying  from  0  to  255. 

(c)  Generate  the  parameter  and  arrays  as  required  by  RSAM;  the  actual  values  used 
are:  ITHV  =  70,  L0  =  4,  NTAREA  =  12o.  NTINT(1)=  70.  NTINT(ntarea)  = 
70,  then  call  the  subroutine  RSAM,  and  after  return  check  for  error  codes. 

(d)  Check  the  contents  of  ever.'  region  generated,  and  decide  if  it  is  too  small  to  be 
considered  as  noise;  assign  to  it  one  of  the  possible  ascii  codes. 

(e)  Obtain  the  tone  of  the  region  and  write  it  in  the  tone  file  to  be  consulted  by  the 
Prolog  Interpreter. 

i.    Exception  handling 

This  program  checks  for   error  codes  generated  by  RSAM;  check  if  the  region 
number  exceeds  200,  which  is  the  maximum  value;  and  provide  with  error  messages, 
j.      Calling  routines 

This  program  is  a  preparatory  step  previous  to  the  invokation    of  the  Change 
Detection  System. 

k.      Called  routines 

RSAM:  Segmentation  routine  from  the  Spider  Library. 

1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 
None. 
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4.        PROGRAM  (FORTRAN)  'SHDPRO' 

a.  Functions  performed 

This  program  will  generate:  a  list  of  regions  intersecting  the  projected  shadow 
area,  and  their  mean  tone  values:  to  do  it  .  this  subroutine  will  use  the  vertices  of  the 
node  and  the  projected  shadow  points  that  were  generated  by  the  subroutine 
SHADOW;  this  list  of  regions  and  tones  will  be  used  by  the  shadow  certainty  rule  of 
the  Change  Detection  System. 

b.  Input 

I\_ARGS:  The  number  of  arguments  received  through  mailbox. 

NPS:  The  number  of  vertices  in  a  polygon. 

Note  that  the  received  coordinate  buffers  are  composed  of  4  elements,  the 
order  used  is:  2-D  coordinates  of  the  projected  image  and  2-D  coordinates  of  the 
predicted  shadow  points. 

c.  Output 

The  output  parameters  are  send  out  via  mailbox  interprocess  communication 


The  average  gray  value  of  the  intersection  area. 
The  number  of  pixels  related  to  the  intersection  area. 
The  output  buffer  to  be  sent  through  mailbox. 
The  number  of  ar2uments  beins  send  out. 


scheme. 

ITONE: 

NPOINTS: 

OBUF: 

NARGS_OUT: 
d.  Common  area 

The  common  area  called  BOARD  has  been  developed  to  provide  interprocess 
communication  through  a  background  storage  and  includes: 

MODEL:  This  matrix  provides  the  'model'  2-D  projection. 

REGMAP:  Includes  all  the  segmented  regions. 

LR:  Side  of  the  stereo  pair  of  images. 

SUNANG:  Provides  with  the  absolute  sun  direction. 

PLNCOEF:  Are  the  base  plane  coelFicients  A.  B.  C  and  D. 

FILESEQ:  A  value  used  to  identify  the  progress  report  files. 

COEF:  Coordinates  corresponding  to  the  exposure  station. 

NREGS:  Is  the  number  of  regions  identified  in  the  photograph. 

AREA:  The  region's  area. 

PERI:  The  region's  perimeter. 

COMPACT:  Parameter  that  show  a  generated  shape  relation  of  the  region. 

MTONE:  The  average  tone  of  each  region. 


54 


VTONE: 

MBR: 
DEBUG: 
IMAGE: 
e.  Work,  area 
STRBUF: 
NPS_SHD: 
INUM: 
MBR_QUAD: 
PREDREG: 
POLYGON: 
MBR2D: 

FLAG  1,2: 

NO_REGS: 

NP_PREDREG: 

VERTEX: 

POINT: 

CP: 

XSOL1.2: 

YSOL1.2 

A,  B,  C: 

QUADLTL: 

NTEST: 


The  tone  variance  of  every  region. 
The  minimum  boundary  rectangle  for  every  region. 
The  type  of  debug  process  under  execution  (0,1  or  2). 
The  digitized  image  of  the  aerial  photograph. 

A  buffer  to  hold  the  input  data  for  the  program. 
The  number  of  points  in  the  calculated  shadow  area. 
Number  of  elements   returned  by  subroutine  CONVERT. 
Is  the  MBR  that  holds  the  QUADLTL  polygon. 
Buffer  to  hold  the  total  calculated  shadow  area. 
Buffer  to  hold  the  coordinates  that  form  the  projected  node. 
Hold  the  projected  vertices  and  the  shadow  pro- 
jection MBR  area  generated  by  shadow  process. 
Are  logical  values  generated  by  the  logical  function  PLUMB, 
and  indicate  wether  a  point  is  in  or  out  a  polygon. 
The  number  of  regions  selected  as  candidate  shadow 
projection  regions. 

The  number  of  points  in  the  total  calculated  shadow 
area  generated  by  the  polygon. 

It  holds  the  projected  edge  coordinates  and  predicted  shadow 
coordinates  from  a  polygon. 

Temporary7  area  used  by  subroutine  CONVERT  to  transfer 
its  output  results. 

Buffer  to  hold  the  central  points  of  the  lines  joining 
adjacent  edge  coordinates  or  shadow  projection  points. 
Coordinates  in  X  at  a  distance  DIST  of  the  central 
point  as  explained  in  CP. 

Coordinates  in  Y  at  a  distance  DIST  of  the  central 
point  as  explained  in  CP. 

Are  the  line  coefficients  of  the  line  joining  two  adjacent 
projected  vertices  or  predicted  shadow  coordinates. 
Is  an  area  formed  by  the  predicted  shadow  coordinates  and 
the  node's  edge  projection  points  that  originated  them. 
Is  the  number  of  points  that  have  been  generated  in  the 
predicted  shadow  projection  process. 
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JJNEIGH:  Offset  to  enlarge  1  pixel  around  the  total  calculated  shadow 

area. 
\IBR_INT:  MBR  to  hold  the  intersection  of  the  shadow  area  with  a 

specified  region. 
RATIO:  Is  the  number  of  points  in  the  shadow  intersection  to  the 

number  of  points  in  the  total  shadow  ratio. 
BUF2D:  Buffer  to  hold  the  calculated  shadow  projection  from  a 

node's  edge. 

f.  Parameters  and  modes 

As  parameters  used  by  this  subroutine,  we  find  the  following:  DIST  is  the 
distance  from  a  calculated  point  perpendicular  to  the  central  point  in  a  line  joining  two 
vertices,  this  is  initialized  to  0.1;  MIN_RATIO:  is  the  value  used  to  determine  if  the 
intersected  part  of  a  shadow  and  a  specific  region  is  large  enough,  this  value  is 
initialized  to  0.1;  and  MAX_BOUNDARY,  is  the  number  of  points  necessary  to 
trigger  the  analysis  of  a  specific  region  adjacent  to  a  shadow  area,  it  is  initialized  to  3. 

This  program  is  designed  to  work,  with  debug  modes  0  and  1  (mode  2  is  under 
development),  when  running  in  mode  1  there  is  additional  information  generated  which 
will  be  displayed  in  the  COMTAL  environment  to  show  the  process  progress  report. 
The  program  uses  as  parameters,  ISX  and  ISY  with  values  of  128  which  are  the 
ranges  in  X  and  Y  of  the  aerial  photograph;  MAXREG  is  100  and  it  is  the  limit  of 
regions  to  be  obtained  from  the  segmented  image. 

g.  File  handling 

DEMON.DAT:      Output  file  that  contains  all  process  messages. 
PROJSHD.DAT:    Input  file  that  contains  the  projected  vertices  coordinates 
and  the  predicted  shadow  coordinates, 
h.  Program  structure 

The  program  for  shadow  generation  is  summarized  in  the  following  steps: 

(a)  Initiate  interprocess  communication  through  event  flag  association,  mailbox  and 
logical  unit  selection;  open  the  files  DEMON.DAT  and  PROJSHD.DAT  in  order 
to  write  all  the  development  messages,  and  to  get  the  input  parameters.  Finally 
the  work,  area  is  initialized  by  clearing  the  butlers  to  hold  the  predicted  shadow 
region  and  MBR2D  is  initialized. 

(b)  The  input  parameters  (2-D  vertex  coordinates  and  predicted  shadow  projection) 
are  loaded  into  the  program,  and  the  MBR  that  holds  the  total  node  projection  is 
generated. 
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(c)  The  projected  shadow  is  generated  by  first  calculating  the  central  points  of  both 

predicted  shadow  projection  sides,  and  projected  edge  sides;  this  is  performed  by 
finding  the  point  in  a  vertical  line  that  is  perpendicular  to  the  line  connecting  two 
adjacent  vertices.  If  this  connecting  line  is  not  parallel  to  one  of  the  axis  (X  or 
Y),  then  the  slope  and  the  intersecting  points  will  have  to  be  used  in  order  to  get 
the  line  coefficients  and  by  this,  obtain  the  central  points. 

(d)  Once  the  possible  solutions  are  obtained  by  offseting  the  central  points  to  both 
sides  of  the  connecting  line  by  a  distance  DIST,  then  the  logical  function 
PLUMB  will  provide  with  the  logical  values  FLAG1  and  FLAG2,  which  are  the 
indicators  to  decide  if  the  possible  solution  lies  inside  or  outside  the  polygon 
formed  by  the  vertices  and  the  shadow  projection  points. 

(e)  By  choosing  the  solutions  that  lie  inside  the  polygon  formed  by  the  predicted 
shadow  points,  it  is  possible  to  decide  if  one  of  the  sides  of  the  node  is  generating 
a  real  shadow,  the  method  used  is  to  verify  if  a  central  point  of  an  edge. and  the 
interior  point  of  the  shadow  polygon  lie  in  the  same  side  of  a  line  connecting  two 
adjacent  shadow  projected  points. 

(0  A  MBR  called  MBR2D  that  was  holding  the  node  projection,  now  is  also 
enlarged  to  hold  the  calculated  shadow  projection;  The  calculated  shadow  that  is 
underimposed  by  the  node  projection  is  eliminated  from  the  predicted  region 
area.  In  case  of  working  under  mode  1,  the  shadow  shapes  projected  from  each 
side  of  the  polygon  are  transfered  to  demonstration  files. 

(g)  Now.  the  shadow  shape  is  enlarged  by  one  pixel  in  order  to  eliminate  the 
discrepancies  in  the  compactedness  of  the  shadow  shape. 

(h)  With  the  shape  of  the  shadow  finally  generated,  the  area  is  intersected  with  the 
regions  of  the  labeled  region  map  in  order  to  find  the  intersection  part  belonging 
to  every  different  region.  The  intersected  parts  are  evaluated  according  to  its  size 
to  decide  if  they  are  large  enough  to  be  included  in  the  output  list. 

(h)  The  minimum  bounding  rectangular  areas  are  calculated  for  each  intersection 
part,  and  with  these  limits  is  possible  to  obtain  the  average  tone  of  the 
intersection  part  from  the  reference  image. 

(i)  Finally  the  results  are  transfered  back,  to  the  certainty  shadow  processing  rule  in 
the  change  detection  system  via  mailbox.  At  the  end  of  the  program  execution, 
the  logical  units  used  will  be  freed  and  an  event  flag  is  set  to  indicate  a 
completion  status. 
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i.    Exception  handling 

Not  used, 
j.       Calling  routines 

This  program  is  initiated  by  the  Prolog  Interpreter  during  execution  of  the 
statement  "'Simage'  (shdpro,{},L)"  in  the  "certainty-of-shadow"  rule, 
k.     Called  routines 

JCPUT:  Used  to  calculate  the  CPU  time. 

BLCKBRD:  To  obtain  connection  with  the  blackboard  storage. 

REGFILL:  To  fill  a  shadow  projection  representation. 

QUADEQ:  To  obtain  the  solution  for  a  quadratic  equation. 

CONVERT:  To  transform  the  ascii  to  integer  values. 

PLUMB:  Logical  function  used  to  verify  if  a  point  is  inside 

or  outside  a  given  polygon. 

FI LENA VI:  Used  to  provide  a  different  filename  for  the  progress 

report  process  while  working  in  mode  1. 
1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 

(a)  In  order  to  link  this  program,  the  following  command  will  have  to  be  given: 
Slink  shdpro.blckbrd,  plumb,  quadeq,  ufo_create,  ufo_open  filenam.  jcput, 
projmbr.  projaff,  regfill,  blkbrd.  opt. 

(b)  The  program  assumes  that  sun  light  is  the  only  light  source,  and  that  the 
background  of  a  tested  object  is  composed  of  one  material.  The  certainty  value 
of  shadow  is  proportional  to  the  estimated  shadow  predicted  region  over  the 
average  tone  from  the  intersection  part  of  the  image  with  the  shadow  projection. 

(c)  The  changes  in  this  program  were  related  with  the  detection  of  some  errors:  the 
position  in  the  vertex  buffer  of  the  projected  and  shadow  coordinates  was 
interchanged:  also  the  use  of  the  PREDREG  array  replaced  properly  the  use  of 
the  MODEL  array  in  some  sentences. 

5.        PROGRAM  (FORTRAN)  'CRBKBD' 
a.   Functions  performed 

The  principal  function  of  this  program  is  the  creation  of  the  'blackboard', 
which  is  an  area  defined  for  interprocess  communication  and  the  measurement  of  the 
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region's  properties  from  a  labeled  picture,  which  will  be  stored  in  a  common  area  called 
BOARD:  the  interested  properties  are:  size,  perimeter,  compactedness,  tone  variations 
and  minimum  bounding  rectangular. 

b.  Input 

IN_ARGS:  The  number  of  arguments  received  through  mailbox. 

LBLF1LE:  The  file  that  contains  the  labeled  regions  map. 

IMGFILE:  The  file  that  contains  the  digitized  image. 

c.  Output 

The  common  area  BOARD  is  generated  during  the  blackboard  generation 
process  and  also  the  same  information  is  written  in  a  region  description  file  to  be 
consulted  by  the  Rule  Base  expert. 

d.  Common  area 

The  common  area  called  BOARD  has  been  developed  to  provide  interprocess 
communication  through  a  background  storage  and  includes: 


MODEL: 
REGMAP: 
LR: 

SUNANG: 
PLNCOEF: 
FILESEQ: 
COEF: 
NREGS: 
AREA: 
PERI: 

COMPACT: 
MTONE: 
VTONE: 
MBR: 
DEBUG: 
IMAGE: 
e.  Work  area 
STRBUF: 
PROFILE: 
COMPACT: 


This  matrix  provides  the  'model'  2-D  projection. 

Includes  all  the  segmented  regions. 

Side  of  the  stereo  pair  of  images. 

Provides  with  the  absolute  sun  direction. 

Are  the  base  plane  coefficients  A,  B,  C  and  D. 

A  value  used  to  identify  the  progress  report  files. 

Coordinates  corresponding  to  the  exposure  station. 

Is  the  number  of  regions  identified  in  the  photograph. 

The  region's  area. 

The  region's  perimeter. 

Parameter  that  show  a  generated  shape  relation  of  the  region. 

The  average  tone  of  each  region. 

The  tone  variance  of  even.'  region. 

The  minimum  boundary  rectangle  for  every  region. 

The  type  of  debug  process  under  execution  (0.1  or  2). 

The  digitized  image  of  the  aerial  photograph. 

A  buffer  to  hold  the  input  data  for  the  program. 
A  file  to  hold  the  Prolog  region  description  data. 
Used  to  measure  the  compactedness  of  a  region. 
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f.  Parameters  and  modes 

This  program  is  designed  to  work  with  debug  modes  0  and  1  (mode  2  is  under 
development),  when  running  in  mode  1  there  is  additional  information  generated  which 
will  be  displayed  in  the  COMTAL  environment  to  show  the  process  progress  report. 
The  program  uses  as  parameters,  ISX  and  ISY  with  values  of  12S  which  are  the 
ranges  in  X  and  Y  of  the  aerial  photograph;  MAXREG  is  100  and  it  is  the  limit  of 
regions  to  be  obtained  from  the  segmented  image. 

g.  File  handling 

LBLFILE:  The  file  that  contains  the  labeled  regions  map. 

IMGFILE:  The  file  that  contains  the  digitized  image. 

PROFILE:  A  file  to  hold  the  Prolog  region  description  data. 

DEM  OX. DAT:      Output  file  that  contains  all  information  regarding 

the  development  of  the  change  detection  process. 
PARAM.DAT:       Input  file  created  by  the  Rule  Base  interpreter  and 

provides  with  the  input  parameters  to  blackboard. 
LDES.PRO:  Output  file  to  be  consulted  by  the  Prolog  Interpreter. 

it  contains  the  region  description  information, 
h.  Program  structure 

The  program  for  blackboard  creation  is  summarized  in  the  following  steps: 

(a)  The  sun  angle  and  plane  coefficients  arrays  are  initialized  with  the  corresponding 
data  from  the  sun  direction  and  base  plane  equation.  The  data  for  left  and  right 
side  of  the  stereo  pair,  and  affine  coefficients  are  also  provided  by  the  input 
parameters. 

(b)  If  debug  mode  I  is  in  progress,  then  the  original  region  map  will  be  transfered  to 
a  demonstration  file;  REGMAP  is  also  generated  from  the  labeled  file  and  the 
IMGFILE  is  assigned  to  the  corresponding  image  file  with  the  pixel  values 
transformed  to  integers  from  0  to  255;  the  field  NREGS  in  the  common  area  is 
also  generated  with  the  number  of  regions  in  the  labeled  file. 

(c)  Using  a  set  of  different  subroutines,  the  BOARD  arrays  for  minimum  bounding 
rectangular,  size,  perimeter,  and  tone  are  calculated  and  a  region  description  file 
is  generated. 

(d)  The  data  to  be  fed  through  the  blackboard  for  the  different  fortran  subroutines, 
is  also  written  in  the  Prolog  region  description  file  that  later  will  be  consulted  by 
the  Rule  Base  expert. 
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(e)      At  the  end  of  the  program  execution,  the  logical  units  will  be  freed  and  an  event 
flag  is  set  to  indicate  a  completion  status. 
1.    Exception  handling 
Not  used, 
j.      Calling  routines 

This  program  is  initiated  by  the  Prolog  Interpreter  during  execution  of  the 
statement  "'Simage'(crbkbd,{},X)"  in  the  initial  stage  of  the  Rule  Base  expert, 
k.     Called  routines 

BLKBRD:  Serve  as  the  data  structure  'blackboard'  creation  routine. 

CQLT1:  Generates  the  MBR  for  the  different  regions. 

PRMT2:  Measures  the  perimeter  of  the  specified  regions. 

AREA2:  Measures  the  size  of  the  labeled  regions. 

TONE:  Get  the  mean  tone  and  variance  of  the  different  regions. 

STPT2:  Measures  the  coordinates  of  the  starting  points  on  the 

border  of  a  specified  region. 
FILENAM:  Used  to  provide  a  different  filename  for  the  progress 

report  process  while  working  in  mode  1. 
1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 

(a)  In  order  to  link  this  program,  the  following  command  will  have  to  be  given: 
Slink  crbkbd,  blkbrd,  convert,  ufo_open,  ufo_create,  filenam,  jcput.tone. 
sysSlibrary:spiderlib;  lib. 

(b)  The  process  of  assigning  values  to  the  variables  in  the  'blackboard'  must  be  done 
after  the  file  is  opened,  otherwise  the  values  will  be  reset  to  zero  because  the 
memory  pages  assigned  to  the  'blackboard'  are  demand  zero  pages,  and  are 
initialized  to  0  by  the  operating  system. 

(c)  To  calculate  the  compactedness  of  a  region,  the  relation  of  area  of  the  region 
against  the  area  o[  a  circle  is  used,  as  reference,  the  compactedness  of  a  circle  is 
1. 
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Number  of  input  arguments. 
Coordinates  of  the  reference  point. 

The  reference  point  value  calculated. 
Number  of  Output  arguments. 


6.        PROGRAM  (FORTRAN)  'PTPROJ' 

a.  Functions  performed 

This  program  is  used  to  obtain  the  reference  point  projected  from  3-D  to  2-D 
coordinates  in  order  to  establish  a  ratio  between  the  predicted  image  conditions  and 
the  observed  values  obtained  by  the  average  of  tone  in  the  reference  point  for  better 
accuracy  the  surrounding  pixels  to  the  reference  point  are  also  considered  for  the 
calculations. 

b.  Input 
INNARGS: 
X.  Y,  Z: 

c.  Output 
IMGREF: 
N: 

d.  Common  area 

The  common  area  called  BOARD  has  been  developed  to  provide  interprocess 
communication  through  a  background  storage  and  includes: 

MODEL:  This  matrix  provides  the  'model'  2-D  projection. 

REG  MAP:  Includes  all  the  segmented  regions. 

LR:  Side  of  the  stereo  pair  of  images. 

SUNANG:  Provides  with  the  absolute  sun  direction. 

PLNCOEF:  Are  the  base  plane  coefficients  A.  B,  C  and  D. 

FILESEQ:  A  value  used  to  identify  the  progress  report  files. 

COEF:  Coordinates  corresponding  to  the  exposure  station. 

NREGS:  Is  the  number  of  regions  identified  in  the  photograph. 

AREA:  The  region's  area. 

PERI:  The  region's  perimeter. 

COMPACT:  Parameter  that  show  a  generated  shape  relation  of  the  region. 

MTONE:  The  average  tone  of  each  region. 

VTONE:  The  tone  variance  of  every  region. 

MBR:  The  minimum  boundary  rectangle  for  even.'  region. 

DEBUG:  The  type  of  debug  process  under  execution  (0,1  or  2). 

IMAGE:  The  digitized  image  of  the  aerial  photograph. 
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e.  Work  area 

UFO_CREATE:     Logical  unit  to  be  created. 
UFO_OPEN:  Logical  unit  to  be  opened. 

U,  V,  W:  Real  equivalent  values  ofX,  Y,  Z. 

IC,  JC:  Coordinate  in  2-D  of  the  reference  point. 

SFORDEF:  Fortran  attribute  flags. 

SSECDEF:  Attribute  flags  for  private,  global  section  creation 

and  mapping. 

f.  Parameters  and  modes 
Not  used. 

g.  File  handling 
Not  used. 

h.  Program  structure 

The  program  for  reference  point  projection  is  summarized  in  the  following 
steps: 

(a)  The  reference  point  projection  program  is  initiated  by  the  Prolog  Interpreter,  and 
the  first  task,  is  to  establish  communication  through  the  memory  management 
interprocess  routine:  event  flag  association,  mailbox  communication  and  logical 
unit  association. 

(b)  The  input  parameters  are  loaded;  the  location  of  the  reference  point  in  2-D  is 
generated. 

(c)  The  average  value  of  the  surrounding  points  to  the  reference  point  included  is 
calculated  from  the  image;  finally  before  returning,  the  logical  units  are  freed  and 
an  event  flag  is  set  to  indicate  a  completion  status. 

i.    Exception  handling 
Not  used, 
j.       Calling  routines 

This  program  is  initiated  by  the  Prolog  Interpreter  by  using  the  command 
'Simage'  (project_mbr.  \IBR,  Proj_MBR),  in  control  rule  No.  2. 
k.      Called  routines 

PROJECT:  To  project  a  3-D  coordinate  to  2-D  coordinate. 

XY2IJ:  Perform  the  afline  transformation. 
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1.       Timing  constraints 

Not  observed, 
m.    Other  considerations 

7.        SUBROUTINE  (FORTRAN)  'AFFIJ' 

a.  Functions  performed 

Perform  the  affine  transformation  (2-D  image  to  2-D  image  6  parameter 
transformation)  includes  a  rotation,  non  perpendicularity  of  the  axes,  two  scale  changes 
and  two  translations. 

b.  Input 

IX, IY:  These  are  the  Image  coordinates  to  be  affined. 

c.  Output 

IX, IY:  Are  the  transformed  2-D  image  coordinates. 

d.  Common  area 
Not  used. 

e.  Work  area 

PHOTO  TO  DIGITAL  IMAGE  TRANSFORMATION: 

Transformation  from  the  photograph  coordinates  to  the  digital  image 
coordinates  is  called  an  affine  transformation  and  represents  a  rotation, 
nonperpendicularity  of  the  axes,  two  scale  changes  and  two  translations.  This  implies 
the  solution  of  the  equations: 

XP=SX(COS(BETA)-DELTA*SIN(BETA))XI-(SY*SIN(BETA))YI  +  Cl 

YP=SX(SIN(BETA)  +  DELTA  *COS(BETA))XI  +  (SY*COS(BETA))YI  +  C2 

where 

XP,  YP    =   Photo  coordinates 

XI,  YI    =   Image  coordinates. 

SX,  SY    =  Scale  changes. 

CI .  C2    =   Translation  in  both  axes. 

DELTA      =   Nonperpendicularity  or  affinity  between  the  axes  X,  Y. 

BETA      =   Rotation  angle. 

A I  =  SX(COS{BETA)  -  DELTA*SIN(BETA)) 

A  2  =  SY(SI.\(BETA)  +  DELTA*COS{BETA)) 

Dl  =  -SY*SIN(BETA) 

B2  =   +SY*COS(BETA) 
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This  transformation  is  also  called  6-parameter  transformation.    This  process 
has  been  implemented  in  this  subroutine  AFFIJ. 

f.  Parameters  and  modes 

g.  File  handling 
Not  used. 

h.  Program  structure 
Not  used. 

The  program  for  afEine  transformation  is  summarized  in  the  following  steps:  . 
(a)      Solve  the  afFine  transformation  equations  to  convert  the  photo  coordinates  to 
display  coordinates, 
i.    Exception  handling 
Not  used, 
j.      Calling  routines 

SELREG:  Select  regions  intersecting  with  the  projected  node. 

SHADOW:  Used  to  generate  the  shadow  projections. 

k.      Called  routines 

Not  used. 
1.      Timing  constraints 

Not  observed. 
m.    Other  considerations 
Not  used. 

8.        SUBROUTINE  (FORTRAN)  'JCPUT' 

a.  Functions  performed 

Obtain  the  CPU  time  consumed  in  a  specific  program. 

b.  Input 

XCPUT:  Variable  to  hold  the  system   CPU  time. 

c.  Output 
Not  used. 

d.  Common  area 
Not  used. 

e.  Work  area 

BUT:  Buffer  to  hold  the  information  required  by  SYSSGETJPI. 
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(a) 


f.  Parameters  and  modes 

Not  used 

g.  File  handling 
Not  used 

h.  Program  structure 

The  program  for  CPU  time  calculation   is  summarized  in  the  following  steps: 
Generate  the  information  necessary  to  call  the  system  utility  SYSSGETJPI. 
i.    Exception  handling 

Not  used. 


j.      Calling  routines 

SELREG: 

SHDPRO: 

REFINE: 
k.     Called  routines 

Not  used. 
1.      Timing  constraints 

Not  observed. 
m.    Other  considerations 

Not  used. 


Select  regions  intersecting  with  the  projected  node. 

Shadow  processing  module. 

To  refine  the  resion's  association  with  the  node. 


9.        SUBROUTINE  (FORTRAN)  'PROJMBR' 

a.  Functions  performed 

This  program  is  used  to  project  the  3-D  MBR  of  the  node  projection  into  2-D 
left  and  right  image  coordinates. 

b.  Input 

Number  of  input  arguments. 
Coordinates  of  the  MBR. 


INNARGS: 
X,  Y,  Z: 

c.  Output 
MBR2D: 

d.  Common  area 
Not  used. 

e.  Work  area 
U,  V,  W: 
I.JTEMP: 
IOLUN: 


The  2-D  coordinates  of  the  MBR. 


Real  equivalent  values  of  X.  Y,  Z. 
Mold  temporarily  the  2-D  MBRs. 
Logical  unit  to  be  used  as  input  output. 
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f.  Parameters  and  modes 

Not  used. 

g.  File  handling 

DEMON.DAT: 

Output  file  that  contains  all  process  messages, 
h.  Program  structure 

The  program  for  minimum  boundary  rectangular  calculation  is  summarized  in 
the  following  steps: 

(a)  The  minimum  boundary  rectangular  program  is  initiated  by  the  Prolog 
Interpreter,  and  the  first  task  is  to  establish  communication  through  the  memory 
management  interprocess  routine:  event  flag  association,  mailbox  communication 
and  logical  unit  association. 

(b)  The  input  parameters  are  loaded;  the  location  of  MBR  2-D  is  generated, 
i.    Exception  handling 

Not  used, 
j.      Calling  routines 

This  program  is  initiated  by  the  Prolog  Interpreter  by  using  the  command 
'Simage'  (project_mbr,MBR,  Proj_MBR)  in  the  execution  of  control  rule  No.  2. 
k.     Called  routines 

PROJECT:  To  project  a  3-D  coordinate  to  2-D  coordinate. 

XY2IJ:  Perform  the  afifine  transformation. 

BLCKBRD:  To  obtain  connection  with  the  blackboard  storage. 

1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 

Not  used. 

10.      SUBROUTINE  (FORTRAN)  'REFINE' 

a.  Functions  performed 

Refine  the  results  provided  by  a  basic  segmentation  algorithm  according  to  the 
intersection  part  of  the  area  mapped  from  a  'model'  and  the  regions  of  a  labeled  image. 
The  refine  process  implies  the  creation  of  new  regions  from  the  intersected  parts  and 
the  reconfiguration  of  the  whole  region  map  in  order  to  simplify  the  analysis  of  the 
change  detection  process. 
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Region  list  associated  with  a  specific  node. 
Number  of  regions  listed  for  the  chosen  node. 


b.  Input 
REGNO: 
NOREGS: 

c.  Output 

The  common  area  BOARD  is  updated  by  including  the  new  regions  and  its 
properties  after  the  refine  process.  The  elements  modified  are:  REGMAP.  AREA, 
PERI,  COMPACT,  MTONE  and  VTONE. 

d.  Common  area 

The  common  area  called  BOARD  has  been  developed  to  provide  interprocess 
communication  through  a  background  storage  and  includes: 

MODEL:  This  matrix  provides  the  'model'  2-D  projection. 

REGMAP:  Includes  all  the  segmented  regions. 

LR:  Side  of  the  stereo  pair  of  images. 

SUNANG:  Provides  with  the  absolute  sun  direction. 

PLNCOEF:  Are  the  base  plane  coefficients  A.  B.  C  and  D. 

FILESEQ:  A  value  used  to  identify  the  progress  report  files. 

COEF:  Coordinates  corresponding  to  the  exposure  station. 

NREGS:  Is  the  number  of  regions  identified  in  the  photograph. 

AREA:  The  region's  area. 

PERI:  The  region's  perimeter. 

COMPACT:  Parameter  that  show  a  generated  shape  relation  of  the  region. 

MTONE:  The  average  tone  of  each  region. 

VTONE:  The  tone  variance  of  every  region. 

MBR:  The  minimum  boundary  rectangle  for  even."  region. 

DEBUG:  The  type  of  debug  process  under  execution  (0,1  or  2). 

IMAGE:  The  digitized  image  of  the  aerial  photograph. 

e.  Work  area 

M_AREA,  M_PERI,  M_MBR.  M_TONE,  M_VTONE.  M_COMPACT  are 
the  temporary  variables  assigned  to  hold  the  related  information  for  the  node  in 
process  as  explained  previously  in  part  4. 

INTS_AREA,  INTS_PERI.  INTS_TONE,  INTS_YTONE. 

INTS_COMPACT  are  the  temporary  variables  assigned  to  hold  the  related  information 
for  the  intersection  part  as  explained  previously  in  part  4. 
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NEW_AREA,  NEW_PERI,  NEW_TONE,  NEW_VTONE, 

\EW_COMPACT    are    the     temporary    variables    assigned    to     hold    the    related 
information  for  the  new  created  region  as  explained  previously  in  part  4. 

NOREGS:  The  total  number  of  regions  identified  in  REGMAP. 

RECT:  Temporary  area  to  hold  the  MBR  of  the  region  under  test. 

FILESEQ:  A  sequence  to  identify  a  demonstration  file. 

JP:  Array  to  hold  a  region  without  the  intersection  part. 

IP:  Array  to  hold  the  intersection  part. 

RATIO  1:  The  intersection  part  over  the  node  area  ratio. 

RATI02:  The  intersection  part  over  the  region  size  ratio. 

REGLIST:  A  list  of  regions  neighboring  another. 

COUNTX.l:  Variables  to  hold  the  CPU  time  consumed  by  the  program. 

NEIGH_REGS:     The  number  of  regions  neighboring  another. 

BUF2D:  An  area  that  shows  the  merged  regions  for  debug  purposes. 

NCAND:  The  candidate  region  with  the  most  similar  characteristics 

to  the  region  in  process. 

NEW_NREGS:      The  number  of  new  regions  obtained  from  the 
intersection  part  of  a  node. 

NEW_LBL_IMAGE:  An  area  with  new  labels  for  all  the  new 

generated  regions, 
f.    Parameters  and  modes 

The  following  three  variables  must  come  from  the  Rule  Base  interpreter  in  the 
final  version:  LOW_BOL'ND  which  is  initialized  to  5,  is  the  size  of  candidate  noise; 
IUP_BOUND  which  is  initialized  to  30,  is  the  maximum  gray  difference  for  small 
region  merging;  GRAY_DIFF  which  is  initialized  to  10  is  the  maximum  gray  difference 
for  big  region  merging.  It  also  uses  as  a  parameter,  NEWREG,  which  is  initialized  to 
20  and  is  the  maximum  number  of  new  regions  that  could  be  generated  after  the 
splitting  and  merge  process. 

This  program  is  designed  to  work,  with  debug  modes  0  and  1  (mode  2  is  under 
development),  when  running  in  mode  1  there  is  additional  information  generated  which 
will  be  displayed  in  the  COMTAL  environment  to  show  the  process  progress  report. 
The  program  uses  as  parameters,  ISX  and  ISV  with  values  of  12S  which  are  the 
ranges  in  X  and  Y  of  the  aerial  photograph. 
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g.  File  handling 

When  working  in  mode  1,  the  files  DEMFILE'  and  'MSGFILE'  will  be 
opened  by  the  subroutine  FILENAM,  and  closed  by  the  refine  process  after  writing  the 
debug  information. 

DEMON.DAT:       Output  file  that  contains  all  process  messages, 
h.  Program  structure 

The  following  steps  will  be  accomplished  by  the  refine  process: 

(a)  Work  area  variables  and  parameters  are  initialized. 

(b)  For  every  region  that  intersects  the  'model',  there  are  two  new  arrays  generated: 
IP  and  JP  which  respectively  contain  the  intersection  part  and  the  region  without 
it;  this  will  be  used  later  when  deciding  if  an  intersection  part  is  too  small  or  not 
in  relation  to  the  node  or  region  size. 

(c)  The  properties  (area,  perimeter,  compactedness,  gray  differences)  are  measured 
for  the  'model'  and  the  intersection  part;  some  debug  information  is  send  to  the 
demonstration  files. 

(d)  The  size  of  the  intersection  parts  is  analized  in  relation  to  the  size  of  the  region 
and  the  'model':  if  it  is  too  big  in  relation  to  both,  then  this  intersection  part  will 
not  be  considered  for  the  splitting  and  merge  process. 

(e)  If  the  intersection  is  small,  then  it  will  be  splited  according  to  the  shape  of  the 
'model',  and  the  resulting  reeions  will  be  merged  with  the  neighbor  region  that 
have  the  most  similar  characteristics. 

(f)  The  new  properties  will  be  generated  and  the  facts  database  will  be  updated 
properly. 

i.    Exception  handling 
Not  used, 
j.       Calling  routines 

This   subroutine   is   called   by   the    Model   guided   split   and   merge   program 
(MGSM)  with  the  sentence:  call  refine  (REG LIST.  IN_NARGS). 
k.     Called  routines 

JCPUT:  Used  to  calculate  the  CPU  time. 

MEASREG:  Used  to  get  the  new  properties  of  a  group  of  regions. 

NEIGH  REGS:        It  get  all  the  neighbor  regions  from  a  selected  one. 
MEASONE:  Measure  the  area,  tone,  mbr  and  compactedness  of 

the  a  selected  region. 
WRPROP:  Write  properties  of  a  region  in  a  file  to  be  consulted 
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by  the  Change  Detection  process. 
ASSREG:  Is  used  to  generate  new  regions  from  a  spliced  intersection 

part  of  a  node. 
FILENAM:  Used  to  provide  a  different  filename  for  the  progress 

report  process  while  working  in  mode  1. 
I.      Timing  constraints 

Not  observed, 
m.    Other  considerations 

In  order  to  link  this  program,  the  following  command  will  have  to  be  given: 

(a)  Slink  mgsm,  refine,  assreg,  neighreg,  tone,  measone,  measreg,  wrprop,  jcput, 
filenam,  ufo_create,  ufo_open,  blkbrd,  blkbrd,  options,  sysSlibrary:spider.lib 

(b)  A  noise  region  is  defined  as  a  regions  whose  size  is  smaller  than  LOW_BOUND 
and  its  tone  variance  is  greater  than  UP_BOUND. 

(c)  Changes  to  the  program  REFINE  were  made  in  the  way  that  all  the  new  data 
generated  for  the  merged  and  splited  regions  could  be  effectively  transfered  to  the 
database  and  common  area  BOARD,  so  it  could  be  used  appropriately  by  the 
incoming  processes.  Other  changes  performed  to  the  program  are  related  with 
the  insertion  of  the  timing  routines. 

1 1.      SUBROUTINE  (FORTRAN)  'BLKBRD' 

a.  Functions  performed 

The  principal  function  of  this  program  is  to  create  and  map  a  section  in 
memory  to  serve  as  a  data  structure  for  the  'blackboard'  interprocess  communication 
area. 

b.  Input 
Not  used. 

c.  Output 
Not  used. 

d.  Common  area 

The  common  area  called  BOARD  has  been  developed  to  provide  interprocess 
communication  through  a  background  storage  and  includes: 

MODEL:  This  matrix  provides  the  'model'  2-D  projection. 

REG.VIAP:  Includes  all  the  segmented  regions. 

LR:  Side  of  the  stereo  pair  of  images. 

SUNANG:  Provides  with  the  absolute  sun  direction. 
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PLNCOEF:  Are  the  base  plane  coefficients  A,  B,  C  and  D. 

FILESEQ:  A  value  used  to  identify  the  progress  report  files. 

COEF:  Coordinates  corresponding  to  the  exposure  station. 

NREGS:  Is  the  number  of  regions  identified  in  the  photograph. 

AREA:  The  region's  area. 

PERI:  The  region's  perimeter. 

COMPACT:  Parameter  that  show  a  generated  shape  relation  of  the  region. 

MTONE:  The  average  tone  of  each  region. 

VTONE:  The  tone  variance  of  even.'  region. 

MBR:  The  minimum  boundary  rectangle  for  every  region. 

DEBUG:  The  type  of  debug  process  under  execution  (0,1  or  2). 

IMAGE:  The  digitized  image  of  the  aerial  photograph. 

e.  Work,  area 

UFO_CREATE:     Logical  unit  to  be  created. 
UFO_OPEN:  Logical  unit  to  be  opened. 

ADDR:  To  provide  the  limits  of  the  background  area. 

SFORDEF:  Fortran  attribute  (lags. 

SSECDEF:  Attribute  Hags  for  private,  global  section  creation 

and  mapping. 

f.  Parameters  and  modes 
Not  used. 

g.  File  handling 
Not  used. 

h.  Program  structure 

The  program  for  blackboard  creation  is  summarized  in  the  following  steps: 

(a)  The  blackboard  creation  program  is  initiated  by  the  Prolog  Interpreter,  and  the 
first  task  is  to  establish  communication  through  the  memory  management 
interprocess  routine:  event  flag  association,  mailbox  communication  and  logical 
unit  association.  Also  the  input  parameters  are  loaded  from  the  file 
PARAM.DAT  and  the  labeled  and  image  files  are  opened. 

(b)  This  subroutine  uses  the  SYSSCRMPSC  system  utility  to  create  and  map  a 
section  in  the  memory,  it  is  the  actual  blackboard  area  the  one  being  created  and 
mapped. 
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i.    Exception  handling 

Not  used, 
j.       Calling  routines 

This  subroutine  is  called  by  CRBKBD  and  all  the  fortran  programs  that  want 
to  get  the  interprocess  communication  established;  they  are: 

MGSM:  Main  program  of  the  'model'  guided  split  and  merge  process. 

SELREG:  Select  regions  intersecting  with  the  projected  node. 

SHDPRO:  Shadow  processing  module. 

PTPROJ:  Used  to  generate  a  point  projection  (3-D  to  2-D). 

k.     Called  routines 

The  system  utility  LIBSGET_LUN  allocates  one  logical  unit  from  a  system 
process  wide  pool. 

1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 
(a)      Slink    crbkbd,     blkbrd,     convert.     ufo_open,     ufo_create,     filenam,    jcput.tone, 
sysSlibrary:spiderlib  lib. 

12.      SUBROUTINE  (FORTRAN)  'PROJECT' 

a.  Functions  performed 

The  main  function  of  this  subroutine  is  to  perform  a  3-D  earth  coordinate  to 
2-D  image  coordinate  transformation  for  a  given  point  coordinates. 

b.  Input 

X.  Y.  Z:  These  are  the  Object  coordinates. 

c.  Output 

XIMA,  YIMA:       Are  the  transformed  2-D  image  coordinates. 

d.  Common  area 

The  common  area  called  BOARD  has  been  developed  to  provide  interprocess 
communication  through  a  background  storage  and  includes: 

MODEL:  This  matrix  provides  the  'model'  2-D  projection. 

REGMAP:  Includes  all  the  segmented  regions. 

LR:  Side  of  the  stereo  pair  of  images. 

SUNANG:  Provides  with  the  absolute  sun  direction. 

PLNCOEF:  Are  the  base  plane  coefficients  A,  B.  C  and  D. 

FILESEQ:  A  value  used  to  identify  the  progress  report  files. 
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COEF: 
NREGS: 

AREA: 
PERI: 

COMPACT: 
MTONE: 
VTONE: 
MBR: 
DEBUG: 
IMAGE: 
e.  Work  area 
NCHARS: 
FOCUS: 
XO.  YO: 
DENOM: 
OMEGA: 
PHI: 
KAPPA: 
XL,  YL,  ZL: 


Coordinates  corresponding  to  the  exposure  station. 

Is  the  number  of  regions  identified  in  the  photograph. 

The  region's  area. 

The  region's  perimeter. 

Parameter  that  show  a  generated  shape  relation  of  the  region. 

The  average  tone  of  each  region. 

The  tone  variance  of  every  region. 

The  minimum  boundary  rectangle  for  every  region. 

The  type  of  debug  process  under  execution  (0.1  or  2). 

The  digitized  image  of  the  aerial  photograph. 


The  number  of  characters  in  the  node  name. 

Focal  length. 

Fiducial  Center. 

Used  for  simplifying  the  transformation  (denominator). 

Exterior  orientation  parameter:  rotation  about  the  X  axis. 

Exterior  orientation  parameter:  rotation  about  the  Y  axis. 

Exterior  orientation  parameter:  rotation  about  the  Z  axis. 

Exposure  station:  Object  coordinates  of  the  perspective 

center. 
COSOME,  SINOME,  COSPHI,  SINPHI,  COSKAP.  SINKAP  are  respectively 
the  cosine  and  sine  of  the  angles  omega,  phi  and  kappa,  used  as  exterior  orientation 
parameters;  the  angle  units  is  in  radians. 

Mil  to  M33  are  the  Orientation  Matrix  coefficients. 

f.  Parameters  and  modes 

The  interior  orientation  parameters  are:  XO,  YO  and  FOCUS;  the  exterior 
orientation  parameters  are:  XL.  YL,  ZL,  OMEGA,  PHI.  KAPPA. 

This  program  is  designed  to  work  with  debug  modes  0  and  1  (mode  2  is  under 
development),  when  running  in  mode  1  there  is  additional  information  generated  which 
will  be  displayed  in  the  COMTAL  environment  to  show  the  process  progress  report. 

g.  File  handling 

DEMON.DAT:      Output  file  that  contains  all  process  messages. 
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h.  Program  structure 

The    program   for    data    projection   is    summarized    in    the    following    steps: 
following  steps: 

(a)      The  Orientation  Matrix  is  generated  for  every  side  of  the  stereo  pair  image,  and 
the   image  coordinate   system  is   constructed   by   using   the   orientation   matrix 
coefficients,    the   interior    orientation   parameters    and    the   exterior    orientation 
parameters, 
i.    Exception  handling 
Not  used. 


Project  a  MBR  from  3-D  to  2-D  coordinates. 
Select  regions  intersecting  with  the  projected  node. 
Used  to  generate  the  shadow  projections. 
Used  to  generate  a  point  projection  (3-D  to  2-D). 


j.      Calling  routines 

PROJMBR: 

SELREG: 

SHADOW: 

PTPROJ: 
k.     Called  routines 

Not  used. 
1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 

Not  used. 

13.      SUBROUTINE  (FORTRAN)  'XY2IJ' 

a.  Functions  performed 

Perform  the  affine  transformation  (2-D  photograph  to  2-D  image  6  parameter 
transformation)  includes  a  rotation,  non  perpendicularity  of  the  axes,  two  scale  changes 
and  two  translations. 

b.  Input 

X,  Y:  These  are  the  Photograph  coordinates. 

c.  Output 


I,  J: 
d.  Common  area 
Not  used. 


Are  the  transformed  2-D  image  coordinates. 
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e.  Work  area 

PHOTO  TO  DIGITAL  IMAGE  TRANSFORMATION: 

Transformation  from  the  photograph  coordinates  to  the  digital  image 
coordinates  is  called  an  affine  transformation  and  represents  a  rotation, 
nonperpendicularity  of  the  axes,  two  scale  changes  and  two  translations.  This  implies 
the  solution  of  the  equations: 

XP=SX(C0S(BETA)-DELTA;::SIN(BETA))XI-(SY*SIN(BETA))YI  +  C1 

YP=SX(SIN(BETA)  +  DELTA*COS(BETA))XI  +  (SY*COS(BETA))YI  +  C2 

where 

XP,  YP    =  Photo  coordinates 

XI,  YI    =   Image  coordinates. 

SX,  SY    =  Scale  changes. 

CI ,  C2    =   Translation  in  both  axes. 

DELTA      =  Nonperpendicularity  or  affinity  between  the  axes  X,  Y. 

BETA      =   Rotation  angle. 

This  transformation  is  also  called  6-parameter  transformation.  This  process 
has  been  implemented  in  this  subroutine  XY2IJ.  The  following  data  has  been  provided 
for  the  analysis  of  the  aerial  photographs  under  test: 

A I  =  SX(COS(BETA)  -  DELTA*SIN{BETA)) 

A2  =  SY(SIN(BETA)  +  DELTA*COS{BETA)) 

Bl  =  -SY*SIN(BETA) 

B2  =   +SY*COS(BETA) 

f.  Parameters  and  modes 

g.  File  handling 
Not  used. 

h.  Program  structure 

Not  used. 

The  program  for  affine  transformation  is  summarized  in  the  following  steps: 
(a)      Solve  the  affine  transformation  equations  to  convert  the  photo  coordinates  to 
display  coordinates, 
i.    Exception  handling 

Not  used. 
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Project  a  MBR  from  3-D  to  2-D  coordinates. 
Select  regions  intersecting  with  the  projected  node. 
Used  to  generate  the  shadow  projections. 
Used  to  generate  a  point  projection  (3-D  to  2-D). 


j.      Calling  routines 

PROJMBR: 

SELREG: 

SHADOW: 

PTPROJ: 
k.     Called  routines 

Not  used. 
1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 

Not  used. 

14.      SUBROUTINE  (FORTRAN)  'CONVERT' 

a.  Functions  performed 

Convert  a  string  buffer  from  the  character  data  type  to  integer  data  type. 

b.  Input 

The  character  string  containing  the  input  data. 
The  number  of  values  to  be  transfered  back. 


The  list  of  integer  values  to  be  returned. 


STRBUF: 
NUM: 

c.  Output 

LIST: 

d.  Common  area 

The  common  area  called  BOARD  has  been  developed  to  provide  interprocess 
communication  through  a  background  storage  and  includes: 

MODEL:  This  matrix  provides  the  model'  2-D  projection. 

REGMAP:  Includes  all  the  segmented  regions. 

LR:  Side  of  the  stereo  pair  of  images. 

SUNANG:  Provides  with  the  absolute  sun  direction. 

PLNCOEF:  Are  the  base  plane  coefficients  A,  B,  C  and  D. 

FILESEQ:  A  value  used  to  identify  the  progress  report  files. 

COEF:  Coordinates  corresponding  to  the  exposure  station. 

NREGS:  Is  the  number  of  regions  identified  in  the  photograph. 

AREA:  The  region's  area. 

PERI:  The  region's  perimeter. 

COMPACT:  Parameter  that  show  a  generated  shape  relation  of  the  region. 

MTONE:  The  average  tone  of  each  region. 
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VTONE:  The  tone  variance  of  every  region. 

MBR:  The  minimum  boundary  rectangle  for  every  region. 

DEBUG:  The  type  of  debug  process  under  execution  (0,1  or  2). 

IMAGE:  The  digitized  image  of  the  aerial  photograph. 

e.  Work  area 

ISIGN:  The  sign  of  the  output  value. 

NO:  Number  of  elements  that  are  being  generated. 

f.  Parameters  and  modes 

The  variable  ISIGN  has  the  values  T  for  positive  and  '-1'  for  negative  values; 
the  character  buffer  being  transformed  is  72  characters  long. 

g.  File  handling 

DEMON.DAT:      Output  file  that  contains  all  process  messages, 
h.  Program  structure 

The  following  steps  will  be  accomplished  by  the  conversion  process: 
(a)      For  even"  character  in  the  string  buffer,  check,  first  the  sign  of  the  number,  then 
verify  that   the  number  is  an  ascii  code  between   '0'   and   '9'.   check  different 
numbers  by  detecting  the  ','  code  and  finish  when  locating  the  closing  bracket  of 
the  list  of  characters  from  the  input  buffer, 
i.    Exception  handling 

This  program  will  issue  error  messages  when  detecting  incompatible  number  of 
integers. 

j.       Calling  routines 

This  program  is  called  by  issuing  the  command  '  call  convert  (strbuf.  list, 
inum)  '  and  the  following  routines  are  calling  this  program: 
CRBKBD:  Generates  the  blackboard  area. 

SHADOW:  Used  to  generate  the  shadow  projections. 

SELREG:  Select  regions  intersecting  with  the  projected  node. 

SHDPRO:  Shadow  processing  module, 

k.     Called  routines 
1.      Timing  constraints 

Not  observed. 
m.    Other  considerations 
Not  used. 
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15.      SUBROUTINE  (FORTRAN)  'SHADOW 

a.  Functions  performed 

The  function  of  this  program  is  to  find  the  2-D  shadow  coordinate  projected 
from  a  3-D  coordinate  object. 

b.  Input 

X,  Y,  Z:  The  coordinates  of  the  object. 

SUNANG:  Array  that  gives  the  sun  direction. 

PLNCOEF:  Values  for  base  plane  coefficients  A,  B,  C  and  D. 

COEF:  Affine  coefficients. 

LR:  Side  of  the  stereo  pair  of  images. 

c.  Output 

IS,  JS:  The  shadow  image  coordinates. 

d.  Work,  area 

SX.SY.SZ:  Are  the  projected  shadow  coordinates  in  3-D. 

R:  The  distance  from  the  object  to  the  ground. 

e.  Parameters  and  modes 

This  program  is  designed  to  work  with  debug  modes  0  and  1  (mode  2  is  under 
development),  when  running  in  mode  1  there  is  additional  information  generated  which 
will  be  displayed  in  the  COMTAL  environment  to  show  the  process  progress  report. 
The  program  uses  as  parameters,  ISX  and  ISY  with  values  of  128  which  are  the 
ranges  in  X  and  Y  of  the  aerial  photograph. 

f.  File  handling 

When  working  in  mode  1,  the  files  'DEMFILE'  and  'MSGFILE'  will  be 
opened  by  the  subroutine  FILENAM,  and  closed  by  the  refine  process  after  dumping 
the  debug  information. 

DEMOX.DAT:      Output  file  that  contains  all  process  messages. 

g.  Program  structure 

The  following  steps  will  be  accomplished  by  the  shadow  process: 

(a)  This  program  uses  the  plane  coefficients  of  the  building  background  base  in  order 
to  obtain  the  distance  from  the  vertex  to  the  ground  ;  with  this  information,  an 
the  sun  position,  the  estimated  ground  coordinates  where  the  shadow  will  be 
located  are  obtained. 

(b)  Then  this  coordinates  of  the  shadow  in  3-D  are  transformed  in  2-D. 
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h.  Exception  handling 

used, 
i.    Calling  routines 

This  subroutine  is  called  by  SELREG  using  the  command  '  call  shado\v(\,y,z, 
sunang,plncoef.lr,is.js). 


]• 


Called  routines 

PROJECT: 

AFFIJ: 

XY2IJ: 

FILENAM: 


To  project  a  3-D  coordinate  to  2-D  coordinate. 
Generate  additional  afTine  transformation  to  the  image. 
Perform  the  afTine  transformation. 
Used  to  provide  a  different  filename  for  the  progress 
report  process  while  working  in  mode  1. 


Timing  constraints 

Not  observed. 
Other  considerations 

Not  used. 
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SUBROUTINE  (FORTRAN)  'FILENAM' 

a.  Functions  performed 

This  subroutine  generates  a  sequence  of  differents  demonstration  files  included 
under  the  variables  DEMFILE  and  MSGFILE  while  running  the  Change  Detection 
System  under  debug  mode  1. 

b.  Input 

Value  that  shows  the  actual  order  for  file  creation. 


FILESEQ: 
c.   Output 
FILENAM: 


Character  string  buffer  to  be  used  for  opening 
the  Demonstration  and  Message  files. 


d.  Common  area 

Not  used. 

e.  Work  area 
Not  used. 

f.  Parameters  and  modes 
Not  used. 


SO 


g.  File  handling 

The  demonstration  files  with  sequence  DEM001  to  DEM999  and  the  message 
files  with  sequence  \ISG001  to  MSG999  are  created  and  opened  in  the  logical  units  1 
(DEMFILE)  and  11  (MSGFILE)  according  to  the  Change  Detection  process  needs, 
h.  Program  structure 

The  program  for  file  names  generation  is  described  as  follows: 
(a)      Ascii  characters  are  generated  by  translating  the  integer  sequence  FILESEQ  digit 
by  digit,  and  incrementing  the  sequence  previously  for  even'  new  file  needed, 
i.    Exception  handling 
Not  used. 
j.      Calling  routines 

This  program  is  called  when  running  under  mode  debug  '1'  by  the  following 
programs: 

SELREG:  Select  regions  intersecting  with  the  projected  node. 

REFINE:  To  refine  the  region's  association  with  the  node. 

SHDPRO:  Shadow  processing  module. 

CRBKBD:  Generates  the  blackboard  area. 

SHADOW:  Used  to  generate  the  shadow  projections. 

MEASONE:  Measure  the  area,  tone,  mbr  and  compactedness. 

MEASREG:  Used  to  get  the  new  properties  of  a  group  of  regions, 

k.     Called  routines 

Not  used. 
1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 
Not  used 

17.      SUBROUTINE  (FORTRAN)  'QUADEQ' 

a.  Functions  performed 

This  program  finds  the  solution  of  a  quadratic  equation. 

b.  Input 

A,  B,  C:  These  are  the  equation  coefficients. 
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c.  Output 

XI,  X2:  These  are  the  calculated  real  solutions. 

d.  Common  area 
Not  used. 

e.  Work  area 
Not  used. 

DELTA:  The  expression  B**2  -  4*A*C 

f.  Parameters  and  modes 
Not  used. 

g.  File  handling 

DEMON.DAT:      Output  file  that  contains  all  process  messages, 
h.  Program  structure 

(a)  Uses  the  formula  solution  =  (-B  +  sqrt(delta)),'(2*A)  to  find  the  real  roots 
of  the  equation. 
1.    Exception  handling 

This  subroutine  checks  for  no  real  solutions  or  input  coefficient  errors. 
j.       Calling  routines 

This  subroutine  is  called  by  the  following  programs: 

SHDPRO:  Shadow  processing  module, 

k.     Called  routines 

Not  used. 
1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 

Not  used. 

18.      SUBROUTINE  (FORTRAN)  'MEASREG' 

a.  Functions  performed 

The  principal  function  of  this  program  is  to  measure  the  properties  of  a 
labeled  picture,  the  interested  properties  are:  size,  average  gray  level,  perimeter, 
compactedness;  this  data  is  obtained  for  a  group  of  regions. 

b.  Input 

IP:  The  intersected  part. 

IMAGE:  The  file  that  contains  the  image  picture. 

ISX,  ISY:  The  image  dimensions  in  X  and  Y   (use  12S). 
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The  region's  area. 

The  region's  perimeter. 

The  region's  compactedness. 

The  average  tone  of  the  region's  gray  values. 

The  reeion's  tone  variance. 


NREGS:  Number  of  regions  in  the  region  map. 

IRX:  The  index  region  number  (selected  region  for 

the  measurements. 
MAXREG:  Value  used  for  dynamic  dimension  of  the  labeled 

region  buffer. 

c.  Output 
AREA: 
PERI: 

COMPACT: 
MTONE: 
VTONE: 

d.  Common  area 
Not  used. 

e.  Work  area 
Not  used. 

f.  Parameters  and  modes 
Not  used. 

g.  Program  structure 

The  program  for  region  measurement  is  summarized  in  the  following  steps: 
(a)      Using  a  set  of  different  subroutines,  the  different  region  properties  are  calculated 
for  the  specified  regions. 
h.  Exception  handling 

Not  used, 
i.    Calling  routines 

This  subroutine  is  called  by  REFINE,    using  the  command:    call  measreg(ip, 
image,  isx.  isy,  nregs,  im,  area,  peri,  compact,  mbr,  mtone,  vtone,  maxreg). 
j.       Called  routines 

CQLT1:  Generates  the  MBR  for  the  different  regions. 

PRMT2:  Measures  the  perimeter  of  the  specified  regions. 

AREA2:  Measures  the  size  of  the  labeled  regions. 

TONE:  Get  the  mean  tone  and  variance  of  the  different  regions. 

STPT2:  Measures  the  coordinates  of  the  starting  points  on  the 

border  of  a  specified  region. 
FILENAM:  Used  to  provide  a  dilferent  filename  for  the  progress 

report  process  while  working  in  mode  1. 
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k.     Timing  constraints 

Not  observed. 
1.      Other  considerations 

(a)  To  calculate  the  compactedness  of  a  region,  the  relation  of  area  of  the  region 
against  the  area  of  a  circle  is  used,  as  reference,  the  compactedness  of  a  circle  is 
1. 


19. 


(a) 
(b) 


SUBROUTINE  (FORTRAN)  'TONE' 

a.  Functions  performed 

Find  the  mean  and  variance  of  the  gray  level  for  a  specified  region. 

b.  Input 

The  labeled  region  file. 
The  image  file. 

The  image  dimensions  (assigned  a  value  of  128). 
The  region  number  being  processed. 
The  Minimum  Boundary  Rectangular  of  the  region. 


LABEL: 
IMAGE: 
ISX,  ISY: 
REGNO: 
RECT: 

c.  Output 
MEAN: 
VARIANCE: 

d.  Common  area 
Not  used. 

e.  Work  area 
NOPOINTS: 
SUM: 
SUMSQ: 

f.  Parameters  and  modes 
Not  used. 


Average  gray  value  of  the  region. 
Variance  of  the  erav  level  in  the  region. 


Number  of  pixels  integrating  the  region. 

Sum  of  the  image  values. 

Sum  of  the  image  values  squared. 


g.  File  handling 

DEMON.DAT:      Output  file  that  contains  all  process  messages, 
h.  Program  structure 

The  program  for  tone  generation  is  summarized  in  the  following  steps: 
Obtain  the  mean  =  SUM  NOPOINTS. 

Obtain  VARIANCE   =  ((SUMSQ  -  SUM  *  SUM)  NOPOINTS)  (NOPOINTS 
I). 
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i.    Exception  handling 

Not  used, 
j.      Calling  routines 

The  following  routines  call  this  program   using  the  command   call  tone(label. 
image,  isx,  isy,  mean,  variance,  RegNo,  rect). 

CRBKBD:  Generates  the  blackboard  area. 

MEASONE:  Measure  the  area,  tone,  mbr  and  compactedness. 

MEASREG:  Used  to  get  the  new  properties  of  a  group  of  regions, 

k.     Called  routines 

Not  used. 
1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 

Not  used. 


20.      SUBROUTINE  (FORTRAN)  'WRPROP' 

a.  Functions  performed 

To  write  the  new  Region  Properties  into  the  file  NEWPROP.DAT  that  will  be 
consulted  by  the  Prolog  Interpreter. 

b.  Input 


I: 

AREA: 

PERI: 

COMPACT: 

MTONE: 

VTONE: 

c.  Output 
Not  used. 

d.  Common  area 
Not  used. 

e.  Work  area 
Not  used. 


The  region  number. 

The  region's  area. 

The  region's  perimeter. 

The  region's  compactedness. 

The  average  tone  of  the  region's  gray  values. 

The  region's  tone  variance. 
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f.  Parameters  and  modes 

Not  used. 

g.  File  handling 

The  file  NEWPROP.DAT  is  output  file  opened  in  logical  unit  No.  13. 
h.  Program  structure 

The  program  for  tone  generation  is  summarized  in  the  following  steps: 
(a)      The  input  parameters  are  formated  and  written  to  be  loaded  as  new  facts  in  the 
file     NEWPROP.DAT     with     the     following     mnemonics     preceding     them: 
REG_SIZE.  REG_PERI,  REG_C0\1PACT.  REG_TONE. 
1.    Exception  handling 
Not  used, 
j.      Calling  routines 

The  following  programs  call  this  subroutine: 

REFINE:  To  refine  the  region's  association  with  the  node. 

The  command  used  is:  call  wrprop  (i,  area,  peri,  compact,  mtone,  vtone) 
k.     Called  routines 

Not  used. 
1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 

At  present  time  the  facts  for  perimeter  and  compactedness  are  not  being  used. 

21.      SUBROUTINE  (FORTRAN)  'MEASONE' 

a.  Functions  performed 

The  principal  function  of  this  program  is  to  measure  the  properties  of  a 
labeled  picture,  the  interested  properties  are:  size,  average  gray  level,  perimeter, 
compactedness;  this  data  is  obtained  for  a  particular  region. 

b.  Input 

IP:  The  intersected  part. 

IMAGE:  The  file  that  contains  the  image  picture. 

ISX.  ISY:  The  image  dimensions  in  X  and  Y   (use  12S). 

NREGS:  Number  of  regions  in  the  region  map. 

IRN:  The  index  region  number  (selected  region  for 

the  measurements. 
MAXREG:  Value  used  for  dynamic  dimension  of  the  labeled 
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The  region's  area. 

The  region's  perimeter. 

The  region's  compactedness. 

The  average  tone  of  the  region's  gray  values. 

The  region's  tone  variance. 


region  buffer 

c.  Output 
AREA: 
PERI: 

COMPACT: 
MTONE: 
VTONE: 

d.  Common  area 
Not  used. 

e.  Work  area 
Not  used. 

f.  Parameters  and  modes 
Not  used. 

g.  Program  structure 

The  program  for  region  measurement  is  summarized  in  the  following  steps: 
(a)      Using  a  set  of  different  subroutines,  the  different  region  properties  are  calculated 
for  the  specified  region, 
h.  Exception  handling 

Not  used. 
i.    Calling  routines 

This  subroutine  is  called  by  REFINE,    using  the  command:    call  measone  (ip, 
image,  isx,  isy,  nregs,  irn,  area,  peri,  compact,  mbr,  mtone,  vtone,  maxreg). 
j.      Called  routines 

CQLT1:  Generates  the  MBR  for  the  different  regions. 

PRMT2:  Measures  the  perimeter  of  the  specified  regions. 

AREA2:  Measures  the  size  of  the  labeled  regions. 

TONE:  Get  the  mean  tone  and  variance  of  the  different  regions. 

STPT2:  Measures  the  coordinates  of  the  starting  points  on  the 

border  of  a  specified  region. 
FILENAM:  Used  to  provide  a  different  filename  for  the  progress 

report  process  while  working  in  mode  1. 
k.     Timing  constraints 
Not  observed. 
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I.      Other  considerations 

(a)  To  calculate  the  compactedness  of  a  region,  the  relation  of  area  of  the  region 
against  the  area  of  a  circle  is  used,  as  reference,  the  compactedness  of  a  circle  is 
1. 
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SUBROUTINE  (FORTRAN)  'NEIGHREGS' 

a.  Functions  performed 

Generate  a  list  of  region  neighbors  to  a  specified  one. 
h.  Input 

ISX.  ISY:  The  image  coordinate  dimensions  (value  of  128). 

LBL_IMG:  Is  the  labeled  region  map. 

RECT:  Limit  of  search  enclosing  the  selected  region. 

REGNO:  The  mother  region  from  where  new  smaller  regions 

were  generated. 

LABEL:  The  selected  region  for  which  neighbors  regions 

are  search. 

c.  Output 

REGLIST:  List  of  regions  neighboring  a  specified  one. 

NREGS:  Number  of  regions  neighboring  a  specified  one. 

NE\V_LBL_IMG:  The  labeled  map  after  splitting  an  intersection 
part  into  new  several  smaller  regions. 

d.  Common  area 
Not  used. 

e.  Work,  area 

JX.  JY:  Variables  to  hold  the  location  of  search. 

DISPL:  Are  the  displacements  in  the  four  possible 

directions  of  searching  (up,  down,  left,  right). 

f.  Parameters  and  modes 
Not  used. 

g.  File  handling 

DEMON.DAT:       Output  file  that  contains  all  process  messages, 
h.  Program  structure 

The  program  for  neighbor  location  is  summarized  in  the  following  steps: 
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(a)  Get  the  input  parameters  and  use  the  MBR  of  the  region  intersecting  the  'model' 
to  limit  the  search  for  the  regions  not  being  equal  to  the  mother  region,  and  that 
are  surrounding  the  specified  new  region. 

(b)  Search  in  all  directions  to  find  the  neighbor  regions, 
i.    Exception  handling 

Not  used, 
j.      Calling  routines 

This  subroutine  is  called  by  REFINE,    using  the  command:    call  neighregs 
(new_lbl_img.  lbl_img,  isx,  isy,  label,  rect,  nregs,  RegList,  RegNo  ). 
k.     Called  routines 

Not  used. 
1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 


23.      SUBROUTINE  (FORTRAN)  'ASSREG' 
a.  Functions  performed 

Assign  new  region  labels  to  the  regions  formed  by  extracting  an  intersection 
part  from  an  old  region. 


b.  Input 
LABEL: 
RECT: 
ISX,  ISY: 
LBLJMG: 

c.  Output 
NEW_NREGS: 
REGION: 

d.  Common  area 
Not  used. 

e.  Work  area 
TABLE: 

KX,  KY: 

STATUS: 


Original  region  number  being  clipped  off. 
MBR  enclosing  the  region  being  clipped  off. 
The  coordinates  sizes  of  the  image. 
Intersection  area  to  be  relabeled. 

The  number  of  new  regions  being  generated. 
The  new  labeled  intersection  area. 


Array  with  character  values  (T  to  TO')  to  relabel 

the  new  created  regions. 

Stack  locations  for  the  coordinates  of  the  region 

being  analyzed. 

Logical  values  to  mark  the  top  address  of  the  stack; 
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if  top  =  0,  then  status  =  .FALSE.. 

f.  Parameters  and  modes 

Not  used. 

g.  File  handling 

FOR008.DAT:        Output  file  used  for  debug  purposes, 
h.  Program  structure 

The  following  steps  will  be  accomplished  by  the  assign  region  process: 

(a)  Reset  the  new  labeled  area  to  zero  value. 

(b)  Identify  the  intersection  region  to  be  reassigned  and  generate  the  new  labeled 
image  using  different  label  values  for  the  isolated  areas. 

(c)  To  identify  isolated  parts  in  the  splited  image,  ASS  REG  calls  in  a  recursive 
method  to  the  subroutine  GENTEST,  which  test  all  the  boundary  pixels  to  the 
one  being  analyzed,  and  by  this  procedure  establishes  individuality. 

(d)  the  results  are  transfered  back  via  output  parameters,  and  Fortran  file  No.  S  is 
used  for  debugging  purposes;  the  MBR  being  analyzed  will  be  dumped  as  well  as 
the  new  labeled  region  buffer. 

1.    Exception  handling 

This    program    issue    error    messages    when    there    exist    an    stack    overflow 
situation. 

j.      Calling  routines 

This  subroutine  is  called  by  the  REFINE  process  with  the  command:  call 
assreg(  lbl_img,  region,  isx,  isy,  label,  rect,  new_nregs). 
k.      Called  routines 

GEN_TEST:  Test  new  created  regions  for  individuality. 

POP,  PUSH:  Used  to  push  and  pop  values  from  a  local  stack. 

1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 
None. 

24.      SUBROUTINE  (FORTRAN)  'REFLEC 

a.   Functions  performed 

To  modify  the  calculated  tone  assigned  to  every  node  as  a  function  of  the  sun 
ravs  incident  ansle  over  the  node's  surface. 
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b.  Input 
ROOF: 
PLNCOEF: 
SUNANG: 
NODENAME: 
LR: 

NCHARS: 
EXPX.Y.Z: 

c.  Output 

The  calculated 

d.  Common  area 
Not  used. 

e.  Work  area 
ALBEDO: 
EO: 

ES: 

ANGTOT: 

ELETOT: 

SANGLE: 

SELEVA: 

BANGLE: 

BELEVA: 

ROFANG: 

ROFELE: 

OBSANG: 

OBSELE: 

SGRADP,Q: 

RGRADP,Q: 

TEMP: 

All  to  C22: 

A.  B.  C: 

SUNNOR: 

EMERGE: 


Are  the  node  3-D  vertices  coordinates. 
Are  the  base  plane  coefficients. 

Values  that  show  the  sun  rays  direction. 
Name  of  the  node  being  analyzed. 
Side  of  the  stereo  pair  of  images. 
Number  of  characters  in  the  node's  name. 
Object  coordinates  for  the  perspective  center. 

new  tone  is  transfered  back,  via  mailbox. 


Reflectance  factor. 

Source  irradiance. 

Radiance  of  a  hemispherical  uniform  source. 

Acumulative  Node  normal  azimuth. 

Acumulative  Node  normal  elevation. 

Sun  azimuth. 

Sun  elevation. 

Base  normal  azimuth. 

Base  normal  elevation. 

Node  normal  azimuth. 

Node  normal  elevation. 

Observer  azimuth. 

Observer  elevation. 

Sun  gradient  coefficients  P  and  Q. 

Node  gradient  coefficients  P  and  Q. 

Temporary  array  used  to  obtain  the  average  plane  equations. 

Matrix  coefficients  used  to  get  plane's  normal  coefficients. 

Plane  equation  coefficients. 

Incident  angle:  angle  between  sun  direction  and  normal 

to  the  plane  formed  by  the  node  vertices. 

Emergent  angle:  angle  between  observer  direction  and 

normal  to  the  plane  formed  by  the  node  vertices. 
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f.  Parameters  and  modes 

Not  used. 

g.  File  handling 

Not  used. 
h.  Program  structure 

The  following  steps  will  be  accomplished  by  the  reflection  process: 

(a)  Work  area  variables  are  initialized;  the  sun  angle  and  elevation  are  obtained  from 
the  sun  angle  direction  coefficients  and  modified  with  respect  to  the  base  plane 
reference;  the  sun  gradient  is  obtained. 

(b)  Get  the  plane  coefficients  for  the  different  vertex  combination,  to  minimize  the 
average  error  of  the  plane  formed  by  the  node's  vertices.  Generate  the  matrix  to 
calculate  the  coefficients  and  solve  the  coefficient  equation;  once  solved,  obtain 
the  roofs  angle  and  elevation  normal  to  the  plane;  get  the  gradient  of  the  node's 
normal. 

(c)  With  the  sun  data  and  the  normal  data,  calculate  the  angle  between  the  sun  and 
the  normal  of  the  node's  roof,  if  it  is  less  than  90  degrees,  it  means  that  the 
surface  is  being  illuminated  directly  by  the  sun  rays;  get  the  observer  direction, 
angle  and  elevation  and  its  gradient  to  solve  the  emergent  angle  between  the 
observer  position  and  the  normal  to  the  node's  surface  normal. 

(d)  With  the  incident  and  emergent  angle,  calculate  the  tone  of  the  surface  according 
to  the  position  o[  the  node's  surface  normal  including  the  different  effect  caused 
by  the  incident  and  emergent  angles. 

(e)  Transfer  the  parameters  back  to  the  calling  program  which  will  be  sended  to  the 
Prolog  Interpreter  via  mailbox. 

i.    Exception  handling 

This  program  checks  for   error  codes  generated  by  RSAM;  check  if  the  region 
number  exceeds  200,  which  is  the  maximum  value:  provide  with  error  messages, 
j.       Calling  routines 

This  program  is  called  by   SELREG    with  the  command  call  reflec(  roof.  tone, 
plncoef,  sunang,  nodename,  lr,  nchars). 
k.     Called  routines 

ANGEL:  Used  to  calculate  the  normal's  azimuth  and  elevation. 

DIRECT:  To  obtain  direction  from  the  sun  and  from  observer. 
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1.      Timing  constraints 

Not  observed, 
m.    Other  considerations 

None. 

25.      FUNCTION  (FORTRAN)  'PLUMB' 

a.  Functions  performed 

This  logical  function  test  whether  a  point  is  inside  or  outside  a  polygon. 

b.  Input 

XO.  YO:  The  test  point  coordinates. 

POLYGON:  Array  of  vertices  forming  the  polygon. 

NEDGES:  Number  of  edges  in  the  polygon. 

c.  Output 

The  logical  values  .TRUE,  or  .FALSE,  are  generated  for  the  variable  PLUMB 
corresponding  to  the  situation  if  the  point  is  inside  of  the  polygon. 

d.  Common  area 
Not  used. 

e.  Work  area 

NTNTSEC:  If  the  number  of  intersections  is  odd,  generate 

a  logical  value  of  .TRUE,  otherwise  .FALSE.. 

f.  Parameters  and  modes 
Not  used. 

g.  File  handling 

DE\ION.DAT:      Output  file  that  contains  all  process  messages, 
h.  Program  structure 

The  program  is  summarized  in  the  following  steps: 
(a)      This  program  uses  pure  analytic  geometry  to  find  if  the  point  lies  in  the  same  side 
of  all  the  lines  joining  the  vertices  of  the  polygon, 
i.    Exception  handling 

This  program  gives  warning  messages  if  the  point  being  analyzed,  lies  on  the 
polygon  itself,  or  if  the  plumb  line  runs  through  a  line  joining  two  adjacent  vertices  of 
the  polygon. 


93 


j.       Calling  routines 

This  program  is  called  by  SHDPRO  using  the  statement  call  plumb(  XO,  YO, 
POLYGON.  NEDGES). 
k.      Called  routines 

Not  used. 
1.      Timing  constraints 

Not  observed. 
m.    Other  considerations 

Not  used 

26.      SUBROUTINE  (PASCAL)  'REGFILL' 

a.  Functions  performed 

Filling  the  interior  of  polygons. 

b.  Input 

A  polygon  which  is  represented  by  a  sequence  of  coordinate  pairs  with  the 
following  form  ((X1,Y1),(X2,Y2),(X3,Y3) (Xn.Yn)},  where 

(1)  (X1.YD  =  (Xn.Yn)  and 

(2)  1<  =Xi.Yi<  =(MAXCOL  or  MAXROW). 

c.  Output 

A  labeled  image  where  the  interior  of  a  polygon  is  represented  by  "1" 
REFERENCE:  see  (  Ref.  20  ). 

d.  Common  area 
Not  used. 

e.  Program  structure 

The  following  steps  will  be  accomplished  by  the  region  filling  process: 
Insert  a  node  in  a  list  in  order  (Procedure  InsertNode). 
Begin  the  Main  Program. 

Open  output  labeled  file. 

Clear  edge  table. 

Create  a  Bucket-sorted  Edge  Table. 

1.  Set  y  to  the  smallest  y-coordinate  which  has  an  entry  in  the  ET. 

2.  Initialize  the  AET  (Active  Edge  Table)  to  be  empty. 

3.  Repeat  until  the  AET  and  ET  are  empty. 

3.1  Merge  information  from  ET  bucket  y  with  the  AET.  maintaining 
AET  sort  order  on  x. 
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3.2  Fill  in  desired  pixel  values  on  scan  line  y  by  using  pairs  of 

x  coordinate  from  the  AET. 

3.3  Remove  from  the  AET  those  entries  for  which  y  =  max. 

3.4  For  each  entry  remaining,  in  AET,  replace  x  by  x  +   1  m. 

This  places  the  next  scan  line  intersection  into  each  entry 
in  the  AET. 

3.5  Resort  the  AET  based  on  X-min. 

3.6  Increment  y  by  1. 

While  step  3.  Print  out  the  labeled  picture. 
End. 

f.  Calling  Routines: 

SELREG:  Select  regions  intersecting  with  the  projected  node. 

SHDPRO:  Shadow  processing  module. 

g.  Exception  handling 
Not  used. 

h.  Timing  constraints 

Not  observed. 
i.    Other  considerations 

None. 
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APPENDIX  C 
PROGRAM  REFLEC 


c 

c 

c     October  15,  1986 

c 

c 


subroutine  reflec(roof , tone ,plncoef , sunang,nodename , lr , 
*  nchars) 

c 
c 

implicit  real*8(a-h,o-z) , integer (i-n) 

dimension  roof (4.3) , plncoef (4) , sunang(3) , temp (3 ,3) 

dimension  expx(2) , expy(2) , expz(2) 

integer  tone 

character*8  nodename 
c 

data  expx(l) ,expx(2)/-2693765 .9 , -2695594.2/ 

data  expy  1  ,expy(2  /-4304520 .4, -4303718.9/ 

data  expz(l) , expz(2)/+3859018. 3 ,+3858623 .0/ 
c 

c     work  area  initialization 
c 

pi=4.0*datan(one) 

albedo=1.0 
c     source  irradiance  eO 

e0=153.5 
c     uniform  hemispherical  source  irradiance  es 

es=58.82 

angtot=0.0 

eletot=0.0 
c 

c     obtain  the  sun  angle  and  elevation 
c 

call  direct (s angle , seleva, sunang(l) , sunang(2) , sunang(3) ) 
c 

c     get  the  sun  direction  with  reference  to  the  base  plane 
c 

call  angel (bangle, be leva,plncoef (1) , plncoef (2) , plncoef (3) ) 

sangle=sangle+bangle 

seleva=seleva+beleva 

if (sangle . gt. (2. *pi) )sangle=sangle-(2.*pi) 
c 

c     calculation  of  the  sun  gradient 
c 

sgradp=-l . 0*dsin(sangle)*(l . /dtan(selevaj 

sgradq=-l . 0*dcos (sangle J*(l . /dt an (seleva' 
c 

c     get  all  the  plane  coefficients  for  different  roof 
c     vertex  combinations, 
c 

do  1  =  1,4 

do  j  =  1,3 

do  k  =  1,3 

jJFJ+l-l 

if(jn.gt  4)jj=i^4 
temp ( j , k)=roof (jj ,k) 
enddo   |   k 
enddo   |   j 
c 

c     get  matrix  calculation  parameters 
c 

all=temp(2,2)-temp(l,2) 
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al2=temp(2 , 3)-temp(l ,3' 

a21=temp(3 , 2)-temp( 1 , 2 

a22=temp(3 , 3) -temp ( 1 , 3 

bl2=temp(2 , l)-temp( 1 , 1 

b22=temp(3,l)-temp(l , 1 

bll=al2 

b21=a22 

cll=bl2 

c21=b22 

cl2=all 

c22=a21 
c 

c     get  the  coefficients  of  the  roof  under  consideration 
c 

a=all*a22-a21*al2 

b=bll*b22-b21*bl2 

C=cll*c22-c21*cl2 
c 

c     get  the  angle  and  elevation  of  the  roof 
c 

call  angel(rofang, rofele , a,b, c) 

angtot=angtot+rotang 

eletot=eletot+rofele 
c 

enddo  |  1 
c 

c     average  normal  to  roof  angle  and  elevation 
c 

angtot=angtot/4 . 

eletot=eletot/4. 
c 
c     gradient  of  the  roof 


c 

rgradp=-l . 0*dsin(angtot)*(l . /dtan(eletot^ 

rgradq=-l .0*dcos (angtot)*(l . /dtan(eletot' 
c 

c     calculation  of  the  angle  incident  I  between  the 
c     sun  and  the  normal  to  the  roof 
c 

dnum=l . 0+sgradp*rgradp+sqradq* rgradg 

denol=dsqr£(l .0+sgradp**2+sgradq**2' 

deno2=dsqrt(l  . 0+rgradp**2+rgradq**2' 

sunnor=dacos(dnum/(denol*deno2) ) 
c 

c     if  the  incident  angle  is  greater  than  90  degrees,  then 
c     the  node  is  not  directly  illuminated  by  the  sun  rays 
c 

if  (sunnor.gt. (pi/2.0) )return 
c 

c     calculation  of  the  emergent  angle  E  between  the 
c     exposure  station  and  the  normal  to  the  roof 
c 

call  direct(obsang,obsele , expx(lr) , expy(lr) , expz(lr) ) 
c 

c     get  observer  gradient 
c 

ogradp=-l .0*dsin(obsang)*(l ./dtanfobsele 

ogradq=-l . 0*dcos (obsang)*(l . /dtan(obsele 
c 

c     get  the  angle  E 
c 

dnum=l . 0+oqradp*rgradp+oqradq*rgradq 

denol=dsqr€(l . 0+ogradp**2+ogradq**2' 

deno2=dsqrt(l .0+rgradp**2+rgradq**2' 

erne rge=dacos (dn urn/ (denol*deno2) ) 
c 
c 
c     calculation  of  the  new  tone  for  the  node 
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c 
c 

c 

c 

c     calculating  the  new  tone  for  the  roof 

c 

smc=dble( tone )+( eO/p i)*albedo 
c*dcos(sunnor)+es*albedo*( (1 . +dcos( emerge) )/2 . ) 

tone=]idint(smc) 
c 

c     loading  the  information  in  the  facts  table 
c 
c 

c      open(unit=9  f ile=l newton.dat ' , status= ' new1 ) 
c      write(9, 900 modename ,ntone 
c      write(9,910)nodename ,nodename 
c  900   format^'  smc  tone(m ' , a<nchars> , ' , ' , i3 , ' ) . ' ) 
c  910   format('  smc"("'  ,  a<nchars>  ,  '  ,m'  ,  a<nchars>  ,  '  )  .  '  ) 
c      close  (9) 
c 
c 

return 

end 

subroutine  angel(angulo,elevat, a,b,c) 
c 

c     to  obtain  the  normal  direction  of  a  surface 
c 

implicit  real*8(a-h,o-z)  , integer*4(i-n) 

pi=4.0*datan(one) 

if (c.gt.0. )go  to  90 
90   if (a.ne.O.O)go  to  100 

angulo=0 .0 

go  to  150 
100   angulo=pi/2.0-datan(dabs(b)/dabs(a) ) 
150   if(fa.le.O.O)  .and.  (b.ge.O  .0)  )angulo=pi-angulo 

if((a.le.0.0). and. (b. It .0 .0 ) )angulo=pi+angulo 

if((a.ge.0.0) .and. (b. le .0 .0) )angulo=2.0*pi-angulo 

elevat=pi/2.0-dabs(datan(c/ (dabs(dcos(angulo)xa) ) ) ) 

return 

end 

subroutine  direct (angulo , elevat , a,b,c) 
c 

c     to  obtain  a  direction  given  a  single  coordinate 
c 

implicit  real*8(a-h,o-z) , integer*-^ i-n) 

pi=4.0*datan(one) 

angulo=datan(b/a) +pi 

e  leva  t=dabs(  da  tan  (c/  (dabs  (dsqrt(  a**2+b*'lt2) ) ) ) ) 

return 

end 
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